[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:08 dist-test-slave-fgb3 krb5kdc[17294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:08 dist-test-slave-fgb3 krb5kdc[17294](info): set up 2 sockets
Aug 09 16:11:08 dist-test-slave-fgb3 krb5kdc[17294](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 16:11:09.738617 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:09 dist-test-slave-fgb3 krb5kdc[17294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:09.858256 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40225
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:40225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:09.944183 17316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:09.944387 17316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:09.944408 17316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:09.945513 17316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:09.945554 17316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:09.945565 17316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:09.945576 17316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:09.945586 17316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:09.947220 17316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:40225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:09.947600 17316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:09.947844 17316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:09.954862 17329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:09.955209 17331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:09.955476 17336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:09.955636 17316 server_base.cc:1034] running on GCE node
I20260809 16:11:09.955900 17316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:09.956274 17316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:09.957410 17316 hybrid_clock.cc:648] HybridClock initialized: now 1786291869957390 us; error 65 us; skew 500 ppm
Aug 09 16:11:09 dist-test-slave-fgb3 krb5kdc[17294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291869, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:09.959965 17316 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:09.960610 17316 webserver.cc:460] Webserver started at http://127.16.224.190:40159/ using document root <none> and password file <none>
I20260809 16:11:09.960891 17316 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:09.960943 17316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:09.961090 17316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:09.962239 17316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "5c3e38e98296439eac8904d6a5158287"
format_stamp: "Formatted at 2026-08-09 16:11:09 on dist-test-slave-fgb3"
server_key: "f3d53c7d321d07741c3ced4fa41c96a0"
server_key_iv: "7cf913bd0135b60de89a5d6bd296fae6"
server_key_version: "encryptionkey@0"
I20260809 16:11:09.962613 17316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "5c3e38e98296439eac8904d6a5158287"
format_stamp: "Formatted at 2026-08-09 16:11:09 on dist-test-slave-fgb3"
server_key: "f3d53c7d321d07741c3ced4fa41c96a0"
server_key_iv: "7cf913bd0135b60de89a5d6bd296fae6"
server_key_version: "encryptionkey@0"
I20260809 16:11:09.964378 17316 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:09.965312 17346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:09.965559 17316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:09.965629 17316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "5c3e38e98296439eac8904d6a5158287"
format_stamp: "Formatted at 2026-08-09 16:11:09 on dist-test-slave-fgb3"
server_key: "f3d53c7d321d07741c3ced4fa41c96a0"
server_key_iv: "7cf913bd0135b60de89a5d6bd296fae6"
server_key_version: "encryptionkey@0"
I20260809 16:11:09.965699 17316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:09.981789 17316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:09.983651 17316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:09.983811 17316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:09.988957 17316 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:40225
I20260809 16:11:09.989035 17426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:40225 every 8 connection(s)
I20260809 16:11:09.989598 17316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:09.991063 17427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:09.993705 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17316
I20260809 16:11:09.993795 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:09.994129 17427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287: Bootstrap starting.
I20260809 16:11:09.994481 17282 external_mini_cluster.cc:1428] Setting key d9ff165718372d5e3616c7658e36bc8a
I20260809 16:11:09.995163 17427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:09.995587 17427 log.cc:824] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:09.996647 17427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287: No bootstrap required, opened a new log
I20260809 16:11:10.000761 17427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3e38e98296439eac8904d6a5158287" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40225 } }
I20260809 16:11:10.000954 17427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:10.001078 17427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3e38e98296439eac8904d6a5158287, State: Initialized, Role: FOLLOWER
I20260809 16:11:10.001269 17427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3e38e98296439eac8904d6a5158287" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40225 } }
I20260809 16:11:10.001467 17427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:10.001518 17427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:10.001577 17427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:10.003337 17427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3e38e98296439eac8904d6a5158287" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40225 } }
I20260809 16:11:10.003567 17427 leader_election.cc:302] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c3e38e98296439eac8904d6a5158287; no voters:
I20260809 16:11:10.003819 17427 leader_election.cc:288] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:10.003863 17442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:10.004055 17442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3e38e98296439eac8904d6a5158287, State: Running, Role: LEADER
I20260809 16:11:10.004248 17442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3e38e98296439eac8904d6a5158287" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40225 } }
I20260809 16:11:10.004348 17427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:10.005019 17445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c3e38e98296439eac8904d6a5158287" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3e38e98296439eac8904d6a5158287" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40225 } } }
I20260809 16:11:10.005102 17445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:10.005421 17446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c3e38e98296439eac8904d6a5158287. Latest consensus state: current_term: 1 leader_uuid: "5c3e38e98296439eac8904d6a5158287" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3e38e98296439eac8904d6a5158287" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40225 } } }
I20260809 16:11:10.005789 17446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:10.006148 17450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:10.007795 17450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:10.011991 17450 catalog_manager.cc:1347] Generated new cluster ID: f56455972ee946d6b813c08fce95a92b
I20260809 16:11:10.012050 17450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:10.015681 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:09.995734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.16.224.190:40225)
0809 16:11:09.996025 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:09.996036 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:09.996290 (+ 254us) server_negotiation.cc:407] Connection header received
0809 16:11:10.000721 (+ 4431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:10.000727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:10.000804 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:10.000924 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:10.003875 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.005328 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.006870 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.007379 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.008700 (+ 1321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.011739 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:10.011743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:10.011743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:10.011752 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:10.013687 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.014084 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.014086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.014087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.014251 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.014536 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.014537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.014537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.014756 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:10.014876 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:10.015155 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 16:11:10.015338 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
I20260809 16:11:10.043154 17450 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:10.043737 17450 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:10.046880 17450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c3e38e98296439eac8904d6a5158287: Generated new TSK 0
I20260809 16:11:10.047039 17450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:10.080931 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:40225
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:10.205412 17472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:10.205693 17472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:10.205730 17472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:10.207419 17472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:10.207486 17472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:10.207523 17472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:10.209645 17472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:40225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:10.210006 17472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:10.210299 17472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:10.216090 17479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:10.216284 17472 server_base.cc:1034] running on GCE node
W20260809 16:11:10.216043 17478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:10.216087 17481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:10.216569 17472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:10.216857 17472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:10.218056 17472 hybrid_clock.cc:648] HybridClock initialized: now 1786291870218039 us; error 32 us; skew 500 ppm
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:10.220993 17472 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:10.221550 17472 webserver.cc:460] Webserver started at http://127.16.224.129:41495/ using document root <none> and password file <none>
I20260809 16:11:10.221818 17472 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:10.221879 17472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:10.222007 17472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:10.223382 17472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "066864d35d5648b7ba464e9f29a72bc8"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "43b148f7748e79a3acc7c85d9d0bd9b3"
server_key_iv: "2543e5d7396b709c08984ba853c1f72c"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.223775 17472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "066864d35d5648b7ba464e9f29a72bc8"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "43b148f7748e79a3acc7c85d9d0bd9b3"
server_key_iv: "2543e5d7396b709c08984ba853c1f72c"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.225342 17472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:11:10.226375 17497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.226545 17472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:10.226645 17472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "066864d35d5648b7ba464e9f29a72bc8"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "43b148f7748e79a3acc7c85d9d0bd9b3"
server_key_iv: "2543e5d7396b709c08984ba853c1f72c"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.226723 17472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:10.255338 17472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:10.257346 17472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:10.257473 17472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:10.257741 17472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:10.258136 17472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:10.258225 17472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.258252 17472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:10.258265 17472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.264672 17472 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:37629
I20260809 16:11:10.264729 17664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:37629 every 8 connection(s)
I20260809 16:11:10.265053 17472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:10.266801 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17472
I20260809 16:11:10.266897 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:10.267056 17282 external_mini_cluster.cc:1428] Setting key 699b62dd5ea4538986ede277b721f399
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:10.275862 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.266092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:39569 (local address 127.16.224.190:40225)
0809 16:11:10.266155 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:10.266157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:10.266370 (+ 213us) server_negotiation.cc:407] Connection header received
0809 16:11:10.266997 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:10.266998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:10.267009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:10.267058 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:10.268590 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.269135 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.270405 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.270714 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.270786 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.273191 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:10.273192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:10.273192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:10.273200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:10.274529 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.274911 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.274912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.274913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.274979 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.275205 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.275206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.275206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.275324 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:10.275381 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:10.275662 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 16:11:10.275766 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
I20260809 16:11:10.276026 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.266064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:40225 (local address 127.16.224.129:39569)
0809 16:11:10.266232 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:10.266278 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:10.266896 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:10.267140 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:10.267146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:10.268176 (+ 1030us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:10.268475 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:10.268482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.269222 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.269224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:10.270270 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:10.270273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.270800 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.270801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:10.270939 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.271485 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:10.271495 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:10.273096 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:10.274610 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:10.274618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:10.274621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:10.274843 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:10.275037 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:10.275037 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:10.275038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:10.275149 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:10.275380 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:10.275389 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:10.275539 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 16:11:10.275630 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:11:10.275796 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260809 16:11:10.276832 17667 heartbeater.cc:347] Connected to a master server at 127.16.224.190:40225
I20260809 16:11:10.276933 17667 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:10.277115 17667 heartbeater.cc:511] Master 127.16.224.190:40225 requested a full tablet report, sending...
I20260809 16:11:10.278131 17374 ts_manager.cc:194] Registered new tserver with Master: 066864d35d5648b7ba464e9f29a72bc8 (127.16.224.129:37629)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
I20260809 16:11:10.279019 17374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:39569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:10.293255 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:40225
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:10.376526 17673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:10.376719 17673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:10.376740 17673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:10.377842 17673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:10.377884 17673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:10.377909 17673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:10.379395 17673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:40225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:10.379630 17673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:10.379848 17673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:10.384651 17687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:10.384652 17684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:10.384860 17673 server_base.cc:1034] running on GCE node
W20260809 16:11:10.384693 17682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:10.385125 17673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:10.385335 17673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:10.386462 17673 hybrid_clock.cc:648] HybridClock initialized: now 1786291870386450 us; error 29 us; skew 500 ppm
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:10.388939 17673 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:10.389382 17673 webserver.cc:460] Webserver started at http://127.16.224.130:46571/ using document root <none> and password file <none>
I20260809 16:11:10.389604 17673 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:10.389650 17673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:10.389771 17673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:10.390893 17673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "221201ba67134cb2a8f36d55616f27b3"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "26e9981249a7bd6bb760a8f8f6c34228"
server_key_iv: "042004e09b39e30207c5634d8e4ddd2f"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.391233 17673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "221201ba67134cb2a8f36d55616f27b3"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "26e9981249a7bd6bb760a8f8f6c34228"
server_key_iv: "042004e09b39e30207c5634d8e4ddd2f"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.392480 17673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:10.393165 17700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.393368 17673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:10.393453 17673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "221201ba67134cb2a8f36d55616f27b3"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "26e9981249a7bd6bb760a8f8f6c34228"
server_key_iv: "042004e09b39e30207c5634d8e4ddd2f"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.393512 17673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:10.404373 17673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:10.406595 17673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:10.406733 17673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:10.406965 17673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:10.407265 17673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:10.407305 17673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.407337 17673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:10.407378 17673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.413455 17673 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:39819
I20260809 16:11:10.413489 17895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:39819 every 8 connection(s)
I20260809 16:11:10.413847 17673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:10.418777 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17673
I20260809 16:11:10.418870 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:10.419026 17282 external_mini_cluster.cc:1428] Setting key 0cc3b238638d97419d4a82d2dce96802
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:10.423671 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.414624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:48133 (local address 127.16.224.190:40225)
0809 16:11:10.414696 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:10.414698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:10.414919 (+ 221us) server_negotiation.cc:407] Connection header received
0809 16:11:10.415439 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:10.415441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:10.415452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:10.415516 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:10.416464 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.416974 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.418074 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.418286 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.418372 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.420801 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:10.420802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:10.420803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:10.420810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:10.422220 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.422719 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.422720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.422721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.422790 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.423045 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.423045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.423045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.423155 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:10.423207 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:10.423528 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 16:11:10.423602 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:11:10.423887 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.414599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:40225 (local address 127.16.224.130:48133)
0809 16:11:10.414783 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:10.414836 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:10.415321 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:10.415583 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:10.415587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:10.416158 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:10.416363 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:10.416369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.417087 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.417089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:10.417966 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:10.417969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.418411 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.418412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:10.418536 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.419156 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:10.419167 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:10.420713 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:10.422335 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:10.422342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:10.422345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:10.422634 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:10.422852 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:10.422853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:10.422853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:10.422987 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:10.423211 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:10.423218 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:10.423380 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 16:11:10.423493 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:11:10.423653 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260809 16:11:10.424479 17896 heartbeater.cc:347] Connected to a master server at 127.16.224.190:40225
I20260809 16:11:10.424580 17896 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:10.424757 17896 heartbeater.cc:511] Master 127.16.224.190:40225 requested a full tablet report, sending...
I20260809 16:11:10.425160 17374 ts_manager.cc:194] Registered new tserver with Master: 221201ba67134cb2a8f36d55616f27b3 (127.16.224.130:39819)
I20260809 16:11:10.425544 17374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:48133
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:10.443112 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:40225
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:10.523875 17909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:10.524067 17909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:10.524089 17909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:10.525177 17909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:10.525219 17909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:10.525243 17909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:10.526773 17909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:40225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:10.526986 17909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:10.527199 17909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:10.531859 17917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:10.531867 17920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:10.531944 17922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:10.532058 17909 server_base.cc:1034] running on GCE node
I20260809 16:11:10.532264 17909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:10.532490 17909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:10.533632 17909 hybrid_clock.cc:648] HybridClock initialized: now 1786291870533616 us; error 26 us; skew 500 ppm
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:10.535921 17909 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:10.536324 17909 webserver.cc:460] Webserver started at http://127.16.224.131:41361/ using document root <none> and password file <none>
I20260809 16:11:10.536509 17909 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:10.536556 17909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:10.536679 17909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:10.537690 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "d4354bf0e3c244c080aac344aa716891"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "efd4b06522e8cb405058126b13df8553"
server_key_iv: "3b83b90e4224151455481b6a4616881f"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.538033 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "d4354bf0e3c244c080aac344aa716891"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "efd4b06522e8cb405058126b13df8553"
server_key_iv: "3b83b90e4224151455481b6a4616881f"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.539407 17909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:10.540088 17933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.540235 17909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:10.540323 17909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "d4354bf0e3c244c080aac344aa716891"
format_stamp: "Formatted at 2026-08-09 16:11:10 on dist-test-slave-fgb3"
server_key: "efd4b06522e8cb405058126b13df8553"
server_key_iv: "3b83b90e4224151455481b6a4616881f"
server_key_version: "encryptionkey@0"
I20260809 16:11:10.540381 17909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:10.548151 17909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:10.549856 17909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:10.549976 17909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:10.550166 17909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:10.550520 17909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:10.550557 17909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.550583 17909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:10.550609 17909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:10.556607 17909 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:34559
I20260809 16:11:10.556661 18122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:34559 every 8 connection(s)
I20260809 16:11:10.556959 17909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:10.557554 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17909
I20260809 16:11:10.557633 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:10.557778 17282 external_mini_cluster.cc:1428] Setting key c5fe9a4f08c2e16a7a72384139f5af79
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:10.565327 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.557642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:38569 (local address 127.16.224.190:40225)
0809 16:11:10.557860 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:10.557863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:10.557874 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:10.558397 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:10.558398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:10.558411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:10.558455 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:10.559359 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.559860 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.560797 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.560988 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.561042 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.563050 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:10.563051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:10.563051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:10.563058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:10.564128 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.564473 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.564475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.564475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.564530 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.564730 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.564731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.564732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.564844 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:10.564900 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:10.565149 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 16:11:10.565228 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260809 16:11:10.565474 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.557585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:40225 (local address 127.16.224.131:38569)
0809 16:11:10.557746 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:10.557795 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:10.558283 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:10.558523 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:10.558527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:10.559081 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:10.559266 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:10.559270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.559936 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.559937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:10.560717 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:10.560719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.561054 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.561055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:10.561165 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.561618 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:10.561626 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:10.562967 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:10.564196 (+ 1229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:10.564202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:10.564205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:10.564410 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:10.564581 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:10.564581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:10.564582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:10.564676 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:10.564898 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:10.564904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:10.565032 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 16:11:10.565120 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:11:10.565271 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260809 16:11:10.565914 18125 heartbeater.cc:347] Connected to a master server at 127.16.224.190:40225
I20260809 16:11:10.566002 18125 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:10.566140 18125 heartbeater.cc:511] Master 127.16.224.190:40225 requested a full tablet report, sending...
I20260809 16:11:10.566491 17374 ts_manager.cc:194] Registered new tserver with Master: d4354bf0e3c244c080aac344aa716891 (127.16.224.131:34559)
I20260809 16:11:10.566839 17374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:38569
I20260809 16:11:10.569665 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:10.592674 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.585599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.16.224.129:37629)
0809 16:11:10.585791 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:10.585795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:10.585806 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:10.585830 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:10.585832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:10.585846 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:10.585907 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:10.586418 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.587109 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.587754 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.588049 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.588096 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.590266 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:10.590277 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:10.590277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:10.590284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:10.591770 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.592072 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.592074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.592074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.592123 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.592254 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.592254 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.592255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.592375 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:10.592480 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:10.592514 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 16:11:10.592577 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
W20260809 16:11:10.593171 17538 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43224
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[17294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:10.615146 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:10.608397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43228 (local address 127.16.224.129:37629)
0809 16:11:10.608465 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:10.608467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:10.608536 (+ 69us) server_negotiation.cc:407] Connection header received
0809 16:11:10.608610 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:10.608611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:10.608626 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:10.608672 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:10.609211 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.609708 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.610495 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:10.610703 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:10.610757 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:10.612872 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:10.612874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:10.612874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:10.612882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:10.614086 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.614452 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.614453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.614454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.614508 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:10.614670 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:10.614671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:10.614671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:10.614769 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:10.614885 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:10.614974 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:11:10.615054 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:11:10.615660 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17472
I20260809 16:11:10.631212 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17673
I20260809 16:11:10.644608 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17909
I20260809 16:11:10.652292 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17316
2026-08-09T16:11:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2155 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[18143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[18143](info): set up 2 sockets
Aug 09 16:11:10 dist-test-slave-fgb3 krb5kdc[18143](info): commencing operation
krb5kdc: starting...
W20260809 16:11:11.802176 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:11 dist-test-slave-fgb3 krb5kdc[18143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:11.926367 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43521
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:12.007349 18169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:12.007560 18169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:12.007589 18169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:12.008723 18169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:12.008764 18169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:12.008781 18169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:12.008797 18169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:12.008812 18169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:12.010295 18169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:12.010579 18169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:12.010805 18169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:12.015713 18180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.015735 18178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.015730 18175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:12.016047 18169 server_base.cc:1034] running on GCE node
I20260809 16:11:12.016301 18169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:12.016594 18169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:12.017746 18169 hybrid_clock.cc:648] HybridClock initialized: now 1786291872017727 us; error 35 us; skew 500 ppm
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:12.020249 18169 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:12.020700 18169 webserver.cc:460] Webserver started at http://127.16.224.190:35921/ using document root <none> and password file <none>
I20260809 16:11:12.020932 18169 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:12.020982 18169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:12.021107 18169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:12.022164 18169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "a60c2093d87047e9a174917664dfdea9"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "9f1d3cbf1439b11985a1e29f15b32581"
server_key_iv: "afa885253a7fc26250dae9260269eca7"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.022541 18169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "a60c2093d87047e9a174917664dfdea9"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "9f1d3cbf1439b11985a1e29f15b32581"
server_key_iv: "afa885253a7fc26250dae9260269eca7"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.023821 18169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:12.024565 18188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.024740 18169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:12.024827 18169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "a60c2093d87047e9a174917664dfdea9"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "9f1d3cbf1439b11985a1e29f15b32581"
server_key_iv: "afa885253a7fc26250dae9260269eca7"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.024890 18169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:12.034578 18169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:12.036269 18169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:12.036417 18169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:12.040269 18169 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:43521
I20260809 16:11:12.040305 18288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:43521 every 8 connection(s)
I20260809 16:11:12.040676 18169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:12.041549 18290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:12.043926 18290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9: Bootstrap starting.
I20260809 16:11:12.044494 18290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:12.044752 18290 log.cc:824] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:12.045481 18290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9: No bootstrap required, opened a new log
I20260809 16:11:12.047122 18290 raft_consensus.cc:348] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60c2093d87047e9a174917664dfdea9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43521 } }
I20260809 16:11:12.047264 18290 raft_consensus.cc:374] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:12.047293 18290 raft_consensus.cc:729] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a60c2093d87047e9a174917664dfdea9, State: Initialized, Role: FOLLOWER
I20260809 16:11:12.047423 18290 consensus_queue.cc:260] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60c2093d87047e9a174917664dfdea9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43521 } }
I20260809 16:11:12.047492 18290 raft_consensus.cc:388] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:12.047524 18290 raft_consensus.cc:482] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:12.047569 18290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:12.048082 18290 raft_consensus.cc:504] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60c2093d87047e9a174917664dfdea9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43521 } }
I20260809 16:11:12.048175 18290 leader_election.cc:302] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a60c2093d87047e9a174917664dfdea9; no voters:
I20260809 16:11:12.048329 18290 leader_election.cc:288] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:12.048370 18293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:12.048555 18293 raft_consensus.cc:686] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [term 1 LEADER]: Becoming Leader. State: Replica: a60c2093d87047e9a174917664dfdea9, State: Running, Role: LEADER
I20260809 16:11:12.048565 18290 sys_catalog.cc:564] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:12.048661 18293 consensus_queue.cc:237] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60c2093d87047e9a174917664dfdea9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43521 } }
I20260809 16:11:12.049261 18294 sys_catalog.cc:455] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a60c2093d87047e9a174917664dfdea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60c2093d87047e9a174917664dfdea9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43521 } } }
I20260809 16:11:12.049353 18294 sys_catalog.cc:458] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:12.049592 18309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:12.049716 18295 sys_catalog.cc:455] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a60c2093d87047e9a174917664dfdea9. Latest consensus state: current_term: 1 leader_uuid: "a60c2093d87047e9a174917664dfdea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60c2093d87047e9a174917664dfdea9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43521 } } }
I20260809 16:11:12.049811 18295 sys_catalog.cc:458] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:12.050127 18309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:12.050819 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18169
I20260809 16:11:12.050904 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:12.051064 17282 external_mini_cluster.cc:1428] Setting key b53716953e139b33af8bc8b53f990fab
I20260809 16:11:12.051718 18309 catalog_manager.cc:1347] Generated new cluster ID: b9194bee8c8147f58d4a72507869c42d
I20260809 16:11:12.051769 18309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:12.057972 18309 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:12.058604 18309 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:12.064127 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.051540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.16.224.190:43521)
0809 16:11:12.051675 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.051684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.054550 (+ 2866us) server_negotiation.cc:407] Connection header received
0809 16:11:12.054673 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.054680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.054736 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.054860 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.055485 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.056410 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.057133 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.057593 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.057701 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.060606 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.060611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.060611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.060624 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.062538 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.062910 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.062912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.062913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.063027 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.063196 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.063196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.063197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.063471 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.063619 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.063702 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.063858 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260809 16:11:12.066608 18309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a60c2093d87047e9a174917664dfdea9: Generated new TSK 0
I20260809 16:11:12.066778 18309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:12.093143 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43521
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:12.177176 18329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:12.177385 18329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:12.177417 18329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:12.178629 18329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:12.178669 18329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:12.178704 18329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:12.180186 18329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:12.180428 18329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:12.180631 18329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:12.185411 18346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.185425 18339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.185420 18340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:12.185611 18329 server_base.cc:1034] running on GCE node
I20260809 16:11:12.185811 18329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:12.186022 18329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:12.187157 18329 hybrid_clock.cc:648] HybridClock initialized: now 1786291872187145 us; error 28 us; skew 500 ppm
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:12.189563 18329 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:12.189967 18329 webserver.cc:460] Webserver started at http://127.16.224.129:35175/ using document root <none> and password file <none>
I20260809 16:11:12.190183 18329 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:12.190229 18329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:12.190389 18329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:12.191402 18329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "ed46a18d155115d27b8015c952c3ac14"
server_key_iv: "370c45a3182f44eefa5893f888387061"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.191726 18329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "ed46a18d155115d27b8015c952c3ac14"
server_key_iv: "370c45a3182f44eefa5893f888387061"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.192937 18329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:12.193588 18354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.193745 18329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:12.193830 18329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "ed46a18d155115d27b8015c952c3ac14"
server_key_iv: "370c45a3182f44eefa5893f888387061"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.193895 18329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:12.209455 18329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:12.211287 18329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:12.211436 18329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:12.211647 18329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:12.211951 18329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:12.211987 18329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.212014 18329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:12.212031 18329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.217639 18329 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:40177
I20260809 16:11:12.217681 18526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:40177 every 8 connection(s)
I20260809 16:11:12.217993 18329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:12.218649 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18329
I20260809 16:11:12.218727 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:12.218916 17282 external_mini_cluster.cc:1428] Setting key c76c8ba73f7b3ff851aa3fe378e9863e
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
I20260809 16:11:12.228605 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.218748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:53825 (local address 127.16.224.190:43521)
0809 16:11:12.219120 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.219122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.219135 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:11:12.220528 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.220529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.220539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.220587 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.221389 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.222140 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.223251 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.223541 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.223623 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.225870 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.225871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.225872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.225881 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.227333 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.227692 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.227693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.227694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.227755 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.227993 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.227993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.227994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.228128 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.228202 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.228459 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.228540 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339}
I20260809 16:11:12.228741 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.218711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43521 (local address 127.16.224.129:53825)
0809 16:11:12.218884 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.218933 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.220415 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.220662 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.220667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.221006 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:12.221280 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.221299 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.222259 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.222260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.223148 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.223152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.223621 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.223621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.223725 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.224255 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:12.224264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:12.225776 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:12.227411 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:12.227420 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:12.227422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:12.227636 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:12.227816 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:12.227816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:12.227817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:12.227929 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:12.228187 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:12.228190 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:12.228327 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.228412 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.228550 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260809 16:11:12.229264 18527 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43521
I20260809 16:11:12.229364 18527 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:12.229535 18527 heartbeater.cc:511] Master 127.16.224.190:43521 requested a full tablet report, sending...
I20260809 16:11:12.230086 18225 ts_manager.cc:194] Registered new tserver with Master: b9e2ddfb5ae04d4a9d0057e6ae364af2 (127.16.224.129:40177)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
I20260809 16:11:12.230816 18225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:53825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:12.243180 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43521
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:12.325662 18537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:12.325888 18537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:12.325923 18537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:12.327117 18537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:12.327159 18537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:12.327196 18537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:12.328724 18537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:12.328966 18537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:12.329182 18537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:12.334049 18548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.334044 18551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.334044 18547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:12.334306 18537 server_base.cc:1034] running on GCE node
I20260809 16:11:12.334551 18537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:12.334779 18537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:12.335927 18537 hybrid_clock.cc:648] HybridClock initialized: now 1786291872335909 us; error 27 us; skew 500 ppm
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:12.338371 18537 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:12.338766 18537 webserver.cc:460] Webserver started at http://127.16.224.130:45837/ using document root <none> and password file <none>
I20260809 16:11:12.338969 18537 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:12.339015 18537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:12.339136 18537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:12.340158 18537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "9c1438305fb34075bf47f7709d24b020"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "fddef5753ecbb3a88b048e36dfe91af0"
server_key_iv: "d49e16bd23f902c2c21af2e77f118352"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.340495 18537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "9c1438305fb34075bf47f7709d24b020"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "fddef5753ecbb3a88b048e36dfe91af0"
server_key_iv: "d49e16bd23f902c2c21af2e77f118352"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.341697 18537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:12.342358 18562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.342499 18537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:12.342584 18537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "9c1438305fb34075bf47f7709d24b020"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "fddef5753ecbb3a88b048e36dfe91af0"
server_key_iv: "d49e16bd23f902c2c21af2e77f118352"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.342650 18537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:12.363446 18537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:12.365729 18537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:12.365880 18537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:12.366127 18537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:12.366495 18537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:12.366534 18537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.366566 18537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:12.366586 18537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.372113 18537 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:40261
I20260809 16:11:12.372184 18757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:40261 every 8 connection(s)
I20260809 16:11:12.372484 18537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:12.378572 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18537
I20260809 16:11:12.378661 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:12.378836 17282 external_mini_cluster.cc:1428] Setting key d7f4df5f14e19982a12ea41cf5c330da
I20260809 16:11:12.381742 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.373229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:35763 (local address 127.16.224.190:43521)
0809 16:11:12.373285 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.373287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.373495 (+ 208us) server_negotiation.cc:407] Connection header received
0809 16:11:12.374485 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.374487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.374496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.374537 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.375149 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.375648 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.376635 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.376808 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.376850 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.379026 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.379028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.379028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.379038 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.380472 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.380848 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.380849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.380849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.380913 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.381099 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.381099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.381099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.381231 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.381296 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.381614 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.381678 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:11:12.382006 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.373184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43521 (local address 127.16.224.130:35763)
0809 16:11:12.373361 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.373404 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.374361 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.374611 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.374615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.374878 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:12.375084 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.375089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.375708 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.375709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.376574 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.376577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.376858 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.376858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.376957 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.377404 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:12.377413 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:12.378912 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:12.380557 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:12.380563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:12.380565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:12.380781 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:12.380965 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:12.380965 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:12.380966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:12.381053 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:12.381296 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:12.381302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:12.381482 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.381596 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.381757 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260809 16:11:12.382433 18758 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43521
I20260809 16:11:12.382532 18758 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:12.382709 18758 heartbeater.cc:511] Master 127.16.224.190:43521 requested a full tablet report, sending...
I20260809 16:11:12.383096 18225 ts_manager.cc:194] Registered new tserver with Master: 9c1438305fb34075bf47f7709d24b020 (127.16.224.130:40261)
I20260809 16:11:12.383458 18225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:35763
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:12.403344 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43521
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:12.488687 18766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:12.488902 18766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:12.488926 18766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:12.490109 18766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:12.490160 18766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:12.490190 18766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:12.491765 18766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:12.492014 18766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:12.492275 18766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:12.497370 18774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.497395 18778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:12.497388 18775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:12.497552 18766 server_base.cc:1034] running on GCE node
I20260809 16:11:12.497838 18766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:12.498160 18766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:12.499334 18766 hybrid_clock.cc:648] HybridClock initialized: now 1786291872499313 us; error 33 us; skew 500 ppm
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:12.501883 18766 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:12.502348 18766 webserver.cc:460] Webserver started at http://127.16.224.131:37621/ using document root <none> and password file <none>
I20260809 16:11:12.502569 18766 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:12.502609 18766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:12.502703 18766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:12.503738 18766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "8861772566584de69917607f5601e60c"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "57fd2e3ccd914f3f7a84cf36985abc82"
server_key_iv: "f01e2f42719739af782602ed120c331c"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.504063 18766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "8861772566584de69917607f5601e60c"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "57fd2e3ccd914f3f7a84cf36985abc82"
server_key_iv: "f01e2f42719739af782602ed120c331c"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.505354 18766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:12.506103 18790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.506275 18766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:12.506392 18766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "8861772566584de69917607f5601e60c"
format_stamp: "Formatted at 2026-08-09 16:11:12 on dist-test-slave-fgb3"
server_key: "57fd2e3ccd914f3f7a84cf36985abc82"
server_key_iv: "f01e2f42719739af782602ed120c331c"
server_key_version: "encryptionkey@0"
I20260809 16:11:12.506454 18766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:12.526569 18766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:12.528815 18766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:12.528941 18766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:12.529136 18766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:12.529441 18766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:12.529476 18766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.529498 18766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:12.529522 18766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:12.535434 18766 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:35273
I20260809 16:11:12.535508 18964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:35273 every 8 connection(s)
I20260809 16:11:12.535794 18766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:12.538964 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18766
I20260809 16:11:12.539052 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:12.539203 17282 external_mini_cluster.cc:1428] Setting key 7dd70416e7bb651550aee51cb27096a8
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:12.544677 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.536569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:54351 (local address 127.16.224.190:43521)
0809 16:11:12.536635 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.536637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.536823 (+ 186us) server_negotiation.cc:407] Connection header received
0809 16:11:12.537758 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.537759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.537769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.537830 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.538499 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.539008 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.540259 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.540456 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.540510 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.542602 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.542603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.542604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.542611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.543655 (+ 1044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.544002 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.544002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.544003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.544046 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.544216 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.544217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.544217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.544311 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.544353 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.544555 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.544614 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:11:12.544852 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.536540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43521 (local address 127.16.224.131:54351)
0809 16:11:12.536702 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.536743 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.537667 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.537897 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.537900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.538172 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:12.538423 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.538430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.539097 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.539099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.540180 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.540183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.540524 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.540525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.540654 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.541089 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:12.541096 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:12.542510 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:12.543717 (+ 1207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:12.543723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:12.543725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:12.543947 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:12.544088 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:12.544088 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:12.544089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:12.544176 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:12.544350 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:12.544353 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:12.544466 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.544541 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.544656 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260809 16:11:12.545311 18966 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43521
I20260809 16:11:12.545403 18966 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:12.545549 18966 heartbeater.cc:511] Master 127.16.224.190:43521 requested a full tablet report, sending...
I20260809 16:11:12.545907 18225 ts_manager.cc:194] Registered new tserver with Master: 8861772566584de69917607f5601e60c (127.16.224.131:35273)
I20260809 16:11:12.546286 18225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:54351
I20260809 16:11:12.550855 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:11:12.558343 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.552779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57580 (local address 127.16.224.190:43521)
0809 16:11:12.552840 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.552843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.552948 (+ 105us) server_negotiation.cc:407] Connection header received
0809 16:11:12.552991 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.552992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.553000 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.553043 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.553575 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.554052 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.555031 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.555327 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.555384 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.556462 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.556463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.556464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.556468 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.557482 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.557756 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.557757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.557757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.557798 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.557915 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.557916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.557916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.558049 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.558120 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.558191 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.558249 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:11:12.561023 18225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:12.562934 18225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:12.569923 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.566078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:40177 (local address 127.0.0.1:56006)
0809 16:11:12.566520 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.566529 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.566584 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.566878 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.566878 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.566880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:12.566906 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.566906 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.568090 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.568092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.568862 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.568864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.569630 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.569631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.569708 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.569723 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.569788 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.569850 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":37,"threads_started":1}
I20260809 16:11:12.569931 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.566220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.16.224.129:40177)
0809 16:11:12.566424 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.566429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.566582 (+ 153us) server_negotiation.cc:407] Connection header received
0809 16:11:12.566650 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.566652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.566669 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.566746 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:12.566968 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.568010 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.568944 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.569550 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.569618 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.569676 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.569811 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.569850 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260809 16:11:12.570394 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.565909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:35273 (local address 127.0.0.1:55330)
0809 16:11:12.566406 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.566417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.566471 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.566747 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.566748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.566750 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:12.566811 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.566812 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.567803 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.567804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.569162 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.569164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.570133 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.570134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.570205 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.570209 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.570257 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.570295 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":27,"thread_start_us":63,"threads_started":1}
I20260809 16:11:12.570446 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.566185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.16.224.131:35273)
0809 16:11:12.566609 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.566614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.566625 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:12.566662 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.566664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.566677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.566742 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:12.566863 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.567700 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.569249 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.569964 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.570039 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.570098 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.570279 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.570371 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
I20260809 16:11:12.570879 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.565846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40261 (local address 127.0.0.1:55452)
0809 16:11:12.565999 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.566030 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.566087 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.566415 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.566418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.566421 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:12.566496 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.566499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.568066 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.568067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.569159 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.569163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.570668 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.570670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.570729 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.570734 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.570776 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.570814 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":3,"thread_start_us":49,"threads_started":1}
I20260809 16:11:12.570787 18429 tablet_service.cc:1467] Processing CreateTablet for tablet 38a45bd0707540a493442391163c6e1a (DEFAULT_TABLE table=test-table [id=6fcaf225f483486c887ad0ccebe6b56b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:12.571143 18429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a45bd0707540a493442391163c6e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:12.571198 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.565966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.16.224.130:40261)
0809 16:11:12.566245 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.566250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.566257 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:12.566292 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.566294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.566310 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.566409 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:12.566582 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.567950 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.569249 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.569964 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.570392 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.571058 (+ 666us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.571086 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.571128 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":102,"threads_started":1}
I20260809 16:11:12.571111 18863 tablet_service.cc:1467] Processing CreateTablet for tablet 38a45bd0707540a493442391163c6e1a (DEFAULT_TABLE table=test-table [id=6fcaf225f483486c887ad0ccebe6b56b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:12.571492 18863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a45bd0707540a493442391163c6e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:12.571794 18651 tablet_service.cc:1467] Processing CreateTablet for tablet 38a45bd0707540a493442391163c6e1a (DEFAULT_TABLE table=test-table [id=6fcaf225f483486c887ad0ccebe6b56b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:12.572108 18651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a45bd0707540a493442391163c6e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:12.573975 18992 tablet_bootstrap.cc:492] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2: Bootstrap starting.
I20260809 16:11:12.574756 18992 tablet_bootstrap.cc:654] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:12.574831 18993 tablet_bootstrap.cc:492] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c: Bootstrap starting.
I20260809 16:11:12.575517 18993 tablet_bootstrap.cc:654] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:12.575824 18994 tablet_bootstrap.cc:492] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020: Bootstrap starting.
I20260809 16:11:12.576105 18993 log.cc:824] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:12.576465 18994 tablet_bootstrap.cc:654] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:12.576562 18992 log.cc:824] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:12.577050 18994 log.cc:824] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:12.577302 18992 tablet_bootstrap.cc:492] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2: No bootstrap required, opened a new log
I20260809 16:11:12.577376 18992 ts_tablet_manager.cc:1397] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 16:11:12.577605 18993 tablet_bootstrap.cc:492] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c: No bootstrap required, opened a new log
I20260809 16:11:12.577652 18993 ts_tablet_manager.cc:1397] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:11:12.578896 18994 tablet_bootstrap.cc:492] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020: No bootstrap required, opened a new log
I20260809 16:11:12.578970 18994 ts_tablet_manager.cc:1397] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:11:12.579202 18992 raft_consensus.cc:348] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.579362 18992 raft_consensus.cc:374] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:12.579397 18992 raft_consensus.cc:729] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e2ddfb5ae04d4a9d0057e6ae364af2, State: Initialized, Role: FOLLOWER
I20260809 16:11:12.579533 18992 consensus_queue.cc:260] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.579531 18993 raft_consensus.cc:348] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.579701 18993 raft_consensus.cc:374] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:12.579747 18993 raft_consensus.cc:729] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8861772566584de69917607f5601e60c, State: Initialized, Role: FOLLOWER
I20260809 16:11:12.579807 18992 ts_tablet_manager.cc:1428] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:12.579877 18993 consensus_queue.cc:260] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.579995 18527 heartbeater.cc:503] Master 127.16.224.190:43521 was elected leader, sending a full tablet report...
I20260809 16:11:12.580155 18993 ts_tablet_manager.cc:1428] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:11:12.580153 18966 heartbeater.cc:503] Master 127.16.224.190:43521 was elected leader, sending a full tablet report...
I20260809 16:11:12.580806 18994 raft_consensus.cc:348] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.580982 18994 raft_consensus.cc:374] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:12.581030 18994 raft_consensus.cc:729] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c1438305fb34075bf47f7709d24b020, State: Initialized, Role: FOLLOWER
I20260809 16:11:12.581143 18994 consensus_queue.cc:260] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.581429 18994 ts_tablet_manager.cc:1428] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:11:12.582485 18758 heartbeater.cc:503] Master 127.16.224.190:43521 was elected leader, sending a full tablet report...
W20260809 16:11:12.623023 18760 tablet.cc:2369] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:12.626287 18999 raft_consensus.cc:482] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:12.626415 18999 raft_consensus.cc:504] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.627014 18999 leader_election.cc:288] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c1438305fb34075bf47f7709d24b020 (127.16.224.130:40261), b9e2ddfb5ae04d4a9d0057e6ae364af2 (127.16.224.129:40177)
I20260809 16:11:12.629978 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.627144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40261 (local address 127.16.224.131:42035)
0809 16:11:12.627198 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.627209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.627247 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.627395 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.627396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.627398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:12.627478 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.627480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.628445 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.628446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.629216 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.629219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.629748 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.629750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.629822 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.629826 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.629870 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.629907 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:11:12.629978 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.627228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:42035 (local address 127.16.224.130:40261)
0809 16:11:12.627269 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.627270 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.627291 (+ 21us) server_negotiation.cc:407] Connection header received
0809 16:11:12.627318 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.627319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.627328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.627370 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:12.627560 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.628372 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.629303 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.629675 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.629739 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.629749 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.629883 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.629917 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:12.630102 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.627416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:43927 (local address 127.16.224.129:40177)
0809 16:11:12.627475 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.627476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.627546 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:11:12.627599 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.627600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.627609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.627670 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:12.627777 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.628482 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.629416 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.629798 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.629858 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.629866 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.629992 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.630024 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:11:12.630254 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.627248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:40177 (local address 127.16.224.131:43927)
0809 16:11:12.627486 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:12.627495 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:12.627547 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:12.627674 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:12.627674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:12.627675 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:12.627725 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.627726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.628553 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.628555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.629336 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:12.629338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.629868 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.629868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:12.629934 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.629937 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:12.629981 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:12.630019 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":129,"threads_started":1}
I20260809 16:11:12.630380 18683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a45bd0707540a493442391163c6e1a" candidate_uuid: "8861772566584de69917607f5601e60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c1438305fb34075bf47f7709d24b020" is_pre_election: true
I20260809 16:11:12.630586 18683 raft_consensus.cc:2454] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8861772566584de69917607f5601e60c in term 0.
I20260809 16:11:12.630682 18461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a45bd0707540a493442391163c6e1a" candidate_uuid: "8861772566584de69917607f5601e60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" is_pre_election: true
I20260809 16:11:12.630841 18461 raft_consensus.cc:2454] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8861772566584de69917607f5601e60c in term 0.
I20260809 16:11:12.630894 18795 leader_election.cc:302] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8861772566584de69917607f5601e60c, 9c1438305fb34075bf47f7709d24b020; no voters:
I20260809 16:11:12.631031 18999 raft_consensus.cc:2781] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:12.631078 18999 raft_consensus.cc:482] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:12.631099 18999 raft_consensus.cc:3037] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:12.631778 18999 raft_consensus.cc:504] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.631906 18999 leader_election.cc:288] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [CANDIDATE]: Term 1 election: Requested vote from peers 9c1438305fb34075bf47f7709d24b020 (127.16.224.130:40261), b9e2ddfb5ae04d4a9d0057e6ae364af2 (127.16.224.129:40177)
I20260809 16:11:12.632095 18683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a45bd0707540a493442391163c6e1a" candidate_uuid: "8861772566584de69917607f5601e60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c1438305fb34075bf47f7709d24b020"
I20260809 16:11:12.632118 18461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a45bd0707540a493442391163c6e1a" candidate_uuid: "8861772566584de69917607f5601e60c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2"
I20260809 16:11:12.632167 18683 raft_consensus.cc:3037] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:12.632200 18461 raft_consensus.cc:3037] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:12.632874 18683 raft_consensus.cc:2454] T 38a45bd0707540a493442391163c6e1a P 9c1438305fb34075bf47f7709d24b020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8861772566584de69917607f5601e60c in term 1.
I20260809 16:11:12.632966 18461 raft_consensus.cc:2454] T 38a45bd0707540a493442391163c6e1a P b9e2ddfb5ae04d4a9d0057e6ae364af2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8861772566584de69917607f5601e60c in term 1.
I20260809 16:11:12.633039 18795 leader_election.cc:302] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8861772566584de69917607f5601e60c, 9c1438305fb34075bf47f7709d24b020; no voters:
I20260809 16:11:12.633164 18999 raft_consensus.cc:2781] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:12.633329 18999 raft_consensus.cc:686] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [term 1 LEADER]: Becoming Leader. State: Replica: 8861772566584de69917607f5601e60c, State: Running, Role: LEADER
I20260809 16:11:12.633421 18999 consensus_queue.cc:237] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } }
I20260809 16:11:12.634361 18225 catalog_manager.cc:5495] T 38a45bd0707540a493442391163c6e1a P 8861772566584de69917607f5601e60c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8861772566584de69917607f5601e60c (127.16.224.131). New cstate: current_term: 1 leader_uuid: "8861772566584de69917607f5601e60c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1438305fb34075bf47f7709d24b020" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8861772566584de69917607f5601e60c" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 35273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9e2ddfb5ae04d4a9d0057e6ae364af2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40177 } health_report { overall_health: UNKNOWN } } }
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:12.649858 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.642910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56014 (local address 127.16.224.129:40177)
0809 16:11:12.642975 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.642977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.643066 (+ 89us) server_negotiation.cc:407] Connection header received
0809 16:11:12.643109 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.643110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.643118 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.643162 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.643676 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.644142 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.644859 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.645042 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.645100 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.647342 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.647351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.647352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.647358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.648726 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.649074 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.649078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.649078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.649137 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.649357 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.649358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.649358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.649506 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.649623 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.649726 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.649790 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:12.674150 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.667262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.16.224.129:40177)
0809 16:11:12.667323 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.667325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.667404 (+ 79us) server_negotiation.cc:407] Connection header received
0809 16:11:12.667486 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.667488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.667497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.667543 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.668077 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.668550 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.669239 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.669428 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.669483 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.671628 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.671630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.671630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.671639 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.672892 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.673327 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.673328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.673329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.673406 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.673606 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.673607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.673607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.673731 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.673825 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.673997 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.674066 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[18143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:12.696988 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:12.690053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.16.224.129:40177)
0809 16:11:12.690135 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:12.690137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:12.690210 (+ 73us) server_negotiation.cc:407] Connection header received
0809 16:11:12.690294 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:12.690295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:12.690305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:12.690385 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:12.690953 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.691443 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.692173 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:12.692359 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:12.692423 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:12.694659 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:12.694661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:12.694662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:12.694669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:12.695904 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.696268 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.696270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.696270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.696347 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:12.696572 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:12.696573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:12.696573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:12.696674 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:12.696746 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:12.696856 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 16:11:12.696919 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 16:11:12.697813 18400 tablet_service.cc:2908] Rejecting scan request for tablet 38a45bd0707540a493442391163c6e1a: Uninitialized: clean time has not yet been initialized
I20260809 16:11:12.699043 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18329
I20260809 16:11:12.710366 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18537
I20260809 16:11:12.727052 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18766
I20260809 16:11:12.739132 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18169
2026-08-09T16:11:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2089 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[19022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[19022](info): set up 2 sockets
Aug 09 16:11:12 dist-test-slave-fgb3 krb5kdc[19022](info): commencing operation
krb5kdc: starting...
W20260809 16:11:13.884579 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:13 dist-test-slave-fgb3 krb5kdc[19022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:14.000636 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46631
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:46631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:14.082212 19043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:14.082455 19043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:14.082491 19043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:14.083609 19043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:14.083648 19043 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:11:14.083666 19043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:14.083683 19043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:14.083698 19043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:14.083714 19043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:14.085180 19043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:46631
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:14.085412 19043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:14.085644 19043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:14.090211 19054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:14.090220 19059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:14.090281 19055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:14.090514 19043 server_base.cc:1034] running on GCE node
I20260809 16:11:14.090727 19043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:14.090991 19043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:14.092133 19043 hybrid_clock.cc:648] HybridClock initialized: now 1786291874092119 us; error 30 us; skew 500 ppm
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:14.098381 19043 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:14.098927 19043 webserver.cc:460] Webserver started at http://127.16.224.190:34989/ using document root <none> and password file <none>
I20260809 16:11:14.099175 19043 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:14.099238 19043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:14.099377 19043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:14.100831 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "e552c268f25847f3903b91cce452ea60"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "4363d40a0d59a81d4e57446daeb177bf"
server_key_iv: "33f8a95a814d12df5f04c78be54a715d"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.101263 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "e552c268f25847f3903b91cce452ea60"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "4363d40a0d59a81d4e57446daeb177bf"
server_key_iv: "33f8a95a814d12df5f04c78be54a715d"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.103020 19043 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 16:11:14.103909 19071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.104075 19043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:14.104176 19043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "e552c268f25847f3903b91cce452ea60"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "4363d40a0d59a81d4e57446daeb177bf"
server_key_iv: "33f8a95a814d12df5f04c78be54a715d"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.104254 19043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:14.131316 19043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:14.133880 19043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:14.134085 19043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:14.140530 19043 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:46631
I20260809 16:11:14.140575 19169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:46631 every 8 connection(s)
I20260809 16:11:14.141018 19043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:14.141844 19170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:14.144912 19170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60: Bootstrap starting.
I20260809 16:11:14.145625 19170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:14.145960 19170 log.cc:824] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:14.146136 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19043
I20260809 16:11:14.146221 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:14.146395 17282 external_mini_cluster.cc:1428] Setting key 6949fe2027738237647d6e47849b5d95
I20260809 16:11:14.146736 19170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60: No bootstrap required, opened a new log
I20260809 16:11:14.149951 19170 raft_consensus.cc:348] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e552c268f25847f3903b91cce452ea60" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46631 } }
I20260809 16:11:14.150102 19170 raft_consensus.cc:374] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:14.150135 19170 raft_consensus.cc:729] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e552c268f25847f3903b91cce452ea60, State: Initialized, Role: FOLLOWER
I20260809 16:11:14.150278 19170 consensus_queue.cc:260] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e552c268f25847f3903b91cce452ea60" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46631 } }
I20260809 16:11:14.150381 19170 raft_consensus.cc:388] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:14.150419 19170 raft_consensus.cc:482] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:14.150460 19170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:14.151059 19170 raft_consensus.cc:504] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e552c268f25847f3903b91cce452ea60" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46631 } }
I20260809 16:11:14.151173 19170 leader_election.cc:302] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e552c268f25847f3903b91cce452ea60; no voters:
I20260809 16:11:14.151324 19170 leader_election.cc:288] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:14.151413 19176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:14.151568 19170 sys_catalog.cc:564] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:14.151672 19176 raft_consensus.cc:686] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [term 1 LEADER]: Becoming Leader. State: Replica: e552c268f25847f3903b91cce452ea60, State: Running, Role: LEADER
I20260809 16:11:14.151813 19176 consensus_queue.cc:237] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e552c268f25847f3903b91cce452ea60" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46631 } }
I20260809 16:11:14.152232 19177 sys_catalog.cc:455] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e552c268f25847f3903b91cce452ea60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e552c268f25847f3903b91cce452ea60" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46631 } } }
I20260809 16:11:14.152307 19177 sys_catalog.cc:458] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:14.152482 19179 sys_catalog.cc:455] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e552c268f25847f3903b91cce452ea60. Latest consensus state: current_term: 1 leader_uuid: "e552c268f25847f3903b91cce452ea60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e552c268f25847f3903b91cce452ea60" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46631 } } }
I20260809 16:11:14.152549 19179 sys_catalog.cc:458] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:14.152516 19194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:14.153148 19194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:14.155134 19208 mvcc.cc:205] Tried to move back new op lower bound from 7316651516537835520 to 7316651516526415872. Current Snapshot: MvccSnapshot[applied={T|T < 7316651516537835520}]
I20260809 16:11:14.155872 19194 catalog_manager.cc:1347] Generated new cluster ID: 3198a47f4713416fb5bb39fc2b658f23
I20260809 16:11:14.155982 19194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:14.159363 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.146916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.16.224.190:46631)
0809 16:11:14.147109 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.147120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.147156 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:11:14.148220 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.148231 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.148296 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.148438 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.149200 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.150666 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.151708 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.152360 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.152439 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.155470 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.155474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.155475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.155486 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.157394 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.157773 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.157776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.157776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.157883 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.158078 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.158079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.158079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.158408 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.158552 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.159021 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.159154 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260809 16:11:14.206796 19194 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:14.207687 19194 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:14.216735 19194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60: Generated new TSK 0
I20260809 16:11:14.216985 19194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:14.242870 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46631
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:14.325718 19215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:14.325937 19215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:14.325968 19215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:14.327142 19215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:14.327184 19215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:14.327211 19215 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:14.327234 19215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:14.328686 19215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:46631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:14.328936 19215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:14.329160 19215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:14.334031 19225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:14.334044 19227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:14.334050 19215 server_base.cc:1034] running on GCE node
W20260809 16:11:14.334031 19229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:14.334503 19215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:14.334754 19215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:14.335903 19215 hybrid_clock.cc:648] HybridClock initialized: now 1786291874335891 us; error 29 us; skew 500 ppm
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:14.338302 19215 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:14.338742 19215 webserver.cc:460] Webserver started at http://127.16.224.129:37285/ using document root <none> and password file <none>
I20260809 16:11:14.338953 19215 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:14.339001 19215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:14.339124 19215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:14.340168 19215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "a3499a7cfd6f26c05812712ab2ee2153"
server_key_iv: "b8e023e73d6151fc576464ea45f8aeca"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.340499 19215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "a3499a7cfd6f26c05812712ab2ee2153"
server_key_iv: "b8e023e73d6151fc576464ea45f8aeca"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.341698 19215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:11:14.342355 19241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.342504 19215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.342590 19215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "a3499a7cfd6f26c05812712ab2ee2153"
server_key_iv: "b8e023e73d6151fc576464ea45f8aeca"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.342656 19215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:14.358023 19215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:14.360069 19215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:14.360181 19215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:14.360687 19215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:14.360733 19215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.360765 19215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:14.360781 19215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.367401 19215 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:34181
I20260809 16:11:14.367473 19423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:34181 every 8 connection(s)
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:14.367805 19215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:14.368685 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19215
I20260809 16:11:14.368762 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:14.368918 17282 external_mini_cluster.cc:1428] Setting key 8963b056d7450cea72385b0098c40b79
I20260809 16:11:14.370923 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.361236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:41467 (local address 127.16.224.190:46631)
0809 16:11:14.361317 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.361319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.361514 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:11:14.362021 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.362022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.362037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.362106 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.363185 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.363982 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.365506 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.365814 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.365887 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.368191 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.368193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.368193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.368201 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.369641 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.370038 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.370039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.370039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.370087 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.370334 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.370335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.370336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.370459 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.370509 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.370790 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.370858 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:11:14.371147 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.361185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46631 (local address 127.16.224.129:41467)
0809 16:11:14.361385 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.361428 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.361910 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.362208 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.362212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.362771 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:14.363058 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.363067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.364090 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.364091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.365397 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.365401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.365903 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.365904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.366048 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.366634 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:14.366644 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:14.368081 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:14.369708 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.369717 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.369719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.369975 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.370136 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.370137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.370137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.370265 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.370512 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:14.370519 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:14.370672 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.370780 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.370932 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260809 16:11:14.371680 19424 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46631
I20260809 16:11:14.371770 19424 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:14.371959 19424 heartbeater.cc:511] Master 127.16.224.190:46631 requested a full tablet report, sending...
I20260809 16:11:14.372910 19096 ts_manager.cc:194] Registered new tserver with Master: 2d95a6dafcbb42ec973787eb7d1b1d6c (127.16.224.129:34181)
I20260809 16:11:14.373723 19096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:41467
I20260809 16:11:14.378130 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.372798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:60431 (local address 127.16.224.190:46631)
0809 16:11:14.372875 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.372877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.372933 (+ 56us) server_negotiation.cc:407] Connection header received
0809 16:11:14.373023 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.373024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.373034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.373059 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.373382 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.373888 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.374902 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.375082 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.375136 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.376019 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.376020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.376020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.376026 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.377175 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.377525 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.377526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.377526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.377570 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.377723 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.377724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.377724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.377806 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.377849 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.378021 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.378073 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:11:14.378196 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.372751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46631 (local address 127.16.224.129:60431)
0809 16:11:14.372882 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.372892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.372954 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.373112 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.373112 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.373261 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:14.373315 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.373316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.373979 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.373980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.374825 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.374827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.375154 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.375155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.375231 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.375570 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:14.375575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:14.375942 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:14.377255 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.377256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.377256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.377462 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.377611 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.377612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.377613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.377671 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.377860 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:14.377861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:14.377917 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.377994 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.378097 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:14.394368 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46631
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:14.476549 19438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:14.476754 19438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:14.476776 19438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:14.477874 19438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:14.477914 19438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:14.477945 19438 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:14.477962 19438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:14.479360 19438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:46631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:14.479555 19438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:14.479725 19438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:14.484488 19446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:14.484529 19448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:14.484531 19438 server_base.cc:1034] running on GCE node
W20260809 16:11:14.484498 19444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:14.484834 19438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:14.485067 19438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:14.486199 19438 hybrid_clock.cc:648] HybridClock initialized: now 1786291874486185 us; error 28 us; skew 500 ppm
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:14.488523 19438 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:14.488976 19438 webserver.cc:460] Webserver started at http://127.16.224.130:37483/ using document root <none> and password file <none>
I20260809 16:11:14.489171 19438 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:14.489218 19438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:14.489346 19438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:14.490397 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "961876aca74343ff9116157257ffc885"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "3e3be1770aa05a85dd812f79649d1a1c"
server_key_iv: "df3d739c78d60ff99531b1bdf86176ee"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.490743 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "961876aca74343ff9116157257ffc885"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "3e3be1770aa05a85dd812f79649d1a1c"
server_key_iv: "df3d739c78d60ff99531b1bdf86176ee"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.491968 19438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:14.492631 19463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.492775 19438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:14.492857 19438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "961876aca74343ff9116157257ffc885"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "3e3be1770aa05a85dd812f79649d1a1c"
server_key_iv: "df3d739c78d60ff99531b1bdf86176ee"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.492928 19438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:14.499042 19438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:14.500869 19438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:14.501001 19438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:14.501518 19438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:14.501557 19438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.501585 19438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:14.501607 19438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:14.508673 19438 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:46037
I20260809 16:11:14.509073 19438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:14.509053 19658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:46037 every 8 connection(s)
I20260809 16:11:14.511330 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.501907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:48029 (local address 127.16.224.190:46631)
0809 16:11:14.501972 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.501974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.502180 (+ 206us) server_negotiation.cc:407] Connection header received
0809 16:11:14.502870 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.502871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.502883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.502932 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.503783 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.504302 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.505303 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.505508 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.505564 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.508496 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.508498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.508498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.508507 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.510035 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.510421 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.510423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.510423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.510488 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.510717 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.510718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.510718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.510837 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.510891 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.511158 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.511242 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:11:14.511487 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.501880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46631 (local address 127.16.224.130:48029)
0809 16:11:14.502042 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.502091 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.502751 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.503018 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.503023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.503367 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:14.503666 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.503673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.504397 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.504398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.505217 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.505220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.505578 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.505579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.505711 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.506180 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:14.506189 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:14.508387 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:14.510109 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.510115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.510117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.510354 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.510541 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.510542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.510542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.510649 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.510889 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:14.510896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:14.511033 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.511114 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.511273 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260809 16:11:14.511947 19659 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46631
I20260809 16:11:14.512027 19659 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:14.512199 19659 heartbeater.cc:511] Master 127.16.224.190:46631 requested a full tablet report, sending...
I20260809 16:11:14.512609 19097 ts_manager.cc:194] Registered new tserver with Master: 961876aca74343ff9116157257ffc885 (127.16.224.130:46037)
I20260809 16:11:14.513046 19097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:48029
I20260809 16:11:14.518091 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.512934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46631 (local address 127.16.224.130:57029)
0809 16:11:14.513061 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.513073 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.513136 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.513257 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.513257 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.513424 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:14.513487 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.513489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.514109 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.514110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.514676 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.514678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.515117 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.515118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.515174 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.515544 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:14.515548 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:14.515802 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:14.517238 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.517238 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.517239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.517377 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.517588 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.517589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.517589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.517634 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.517850 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:14.517851 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:14.517885 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.517929 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.518014 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260809 16:11:14.518097 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.513006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:57029 (local address 127.16.224.190:46631)
0809 16:11:14.513048 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.513049 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.513120 (+ 71us) server_negotiation.cc:407] Connection header received
0809 16:11:14.513197 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.513198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.513208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.513253 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.513557 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.514028 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.514742 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.515031 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.515117 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.515921 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.515923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.515923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.515930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.517177 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.517470 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.517471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.517471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.517530 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.517671 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.517671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.517671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.517809 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.517859 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.517950 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.518035 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:11:14.519050 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19438
I20260809 16:11:14.519127 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:14.519291 17282 external_mini_cluster.cc:1428] Setting key 1411cb5d208a70aff7ab05534eb73036
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:14.542972 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46631
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:14.626076 19670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:14.626274 19670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:14.626296 19670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:14.627489 19670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:14.627532 19670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:14.627550 19670 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:14.627574 19670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:14.629029 19670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:46631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:14.629240 19670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:14.629433 19670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:14.634138 19682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:14.634155 19679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:14.634145 19680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:14.634466 19670 server_base.cc:1034] running on GCE node
I20260809 16:11:14.634653 19670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:14.634845 19670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:14.635980 19670 hybrid_clock.cc:648] HybridClock initialized: now 1786291874635958 us; error 40 us; skew 500 ppm
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:14.638258 19670 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:14.638703 19670 webserver.cc:460] Webserver started at http://127.16.224.131:41949/ using document root <none> and password file <none>
I20260809 16:11:14.638912 19670 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:14.638959 19670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:14.639084 19670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:14.640132 19670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "e82a9802236f4255808ef8bc27a7c8df"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "23c94374f233e50652591cadfc793ef1"
server_key_iv: "e1e63d50edd2751f3fd4c3b9a78aa7f0"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.640460 19670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "e82a9802236f4255808ef8bc27a7c8df"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "23c94374f233e50652591cadfc793ef1"
server_key_iv: "e1e63d50edd2751f3fd4c3b9a78aa7f0"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.641650 19670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:14.642294 19693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.642540 19670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:14.642649 19670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "e82a9802236f4255808ef8bc27a7c8df"
format_stamp: "Formatted at 2026-08-09 16:11:14 on dist-test-slave-fgb3"
server_key: "23c94374f233e50652591cadfc793ef1"
server_key_iv: "e1e63d50edd2751f3fd4c3b9a78aa7f0"
server_key_version: "encryptionkey@0"
I20260809 16:11:14.642717 19670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:14.655720 19670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:14.657377 19670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:14.657517 19670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:14.658061 19670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:14.658113 19670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.658142 19670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:14.658172 19670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:14.665309 19670 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:37701
I20260809 16:11:14.665737 19670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:14.666379 19885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:37701 every 8 connection(s)
I20260809 16:11:14.667551 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.658543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:50821 (local address 127.16.224.190:46631)
0809 16:11:14.658608 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.658612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.658846 (+ 234us) server_negotiation.cc:407] Connection header received
0809 16:11:14.659552 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.659554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.659567 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.659619 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.660473 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.661022 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.662489 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.662692 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.662749 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.664843 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.664845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.664845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.664853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.666486 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.666848 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.666849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.666849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.666900 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.667074 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.667075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.667075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.667177 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.667221 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.667421 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.667485 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:11:14.667723 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.658530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46631 (local address 127.16.224.131:50821)
0809 16:11:14.658703 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.658752 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.659436 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.659706 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.659710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.660078 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:14.660367 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.660374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.661092 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.661094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.662409 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.662413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.662793 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.662795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.662921 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.663387 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:14.663397 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:14.664764 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:14.666570 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.666576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.666578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.666790 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.666949 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.666949 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.666950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.667035 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.667212 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:14.667215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:14.667327 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.667402 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.667527 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260809 16:11:14.668006 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19670
I20260809 16:11:14.668097 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:14.668251 17282 external_mini_cluster.cc:1428] Setting key 09e3695ed819cf2c78733687d65314db
I20260809 16:11:14.668299 19886 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46631
I20260809 16:11:14.668378 19886 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:14.668571 19886 heartbeater.cc:511] Master 127.16.224.190:46631 requested a full tablet report, sending...
I20260809 16:11:14.668942 19097 ts_manager.cc:194] Registered new tserver with Master: e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701)
I20260809 16:11:14.669358 19097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:50821
I20260809 16:11:14.674006 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.669293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:35133 (local address 127.16.224.190:46631)
0809 16:11:14.669518 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.669519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.669526 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:14.669543 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.669543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.669552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.669609 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.669968 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.670481 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.671119 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.671288 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.671328 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.672159 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.672160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.672161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.672165 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.673223 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.673497 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.673497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.673498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.673541 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.673675 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.673676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.673676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.673759 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.673801 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.673888 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.673948 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260809 16:11:14.674023 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.669230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46631 (local address 127.16.224.131:35133)
0809 16:11:14.669347 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.669357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.669398 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.669617 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.669618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.669819 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:14.669887 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.669889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.670555 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.670556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.671062 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.671063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.671333 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.671334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.671383 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.671786 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:14.671791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:14.672105 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:14.673284 (+ 1179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.673285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.673285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.673441 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.673583 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:14.673584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:14.673584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:14.673637 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:14.673794 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:14.673794 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:14.673828 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.673871 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.673949 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
I20260809 16:11:14.680440 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:11:14 dist-test-slave-fgb3 krb5kdc[19022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:14.702785 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.696265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.16.224.190:46631)
0809 16:11:14.696327 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.696329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.696338 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:14.696366 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.696367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.696377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.696421 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:14.696972 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.697691 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.698525 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.698726 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.698780 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.700754 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:14.700755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:14.700756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:14.700762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:14.701777 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.702072 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.702073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.702073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.702126 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:14.702276 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:14.702278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:14.702278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:14.702427 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:14.702511 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.702620 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.702694 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:11:14.704178 19097 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:14.705703 19097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:14.711778 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.707894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.16.224.130:46037)
0809 16:11:14.708087 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.708093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.708104 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:14.708145 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.708147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.708163 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.708243 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:14.708469 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.709320 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.710669 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.711382 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.711453 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.711514 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.711652 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.711688 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260809 16:11:14.711757 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.707789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46037 (local address 127.0.0.1:46786)
0809 16:11:14.707907 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.707948 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.708012 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.708319 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.708323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.708325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:14.708384 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.708385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.709400 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.709401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.710573 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.710576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.711456 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.711457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.711548 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.711562 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.711628 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.711679 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"spinlock_wait_cycles":384,"thread_start_us":67,"threads_started":1}
I20260809 16:11:14.712133 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.708184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.16.224.129:34181)
0809 16:11:14.708385 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.708389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.708397 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:14.708445 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.708446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.708460 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.708531 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:14.708729 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.709822 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.710895 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.711541 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.711610 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.711659 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.711999 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.712038 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
I20260809 16:11:14.712215 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.708005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:34181 (local address 127.0.0.1:33922)
0809 16:11:14.708182 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.708191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.708247 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.708599 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.708600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.708604 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:14.708655 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.708656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.709917 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.709919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.710821 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.710823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.711696 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.711697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.711759 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.711763 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.711808 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.711848 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":32,"thread_start_us":38,"threads_started":1}
I20260809 16:11:14.712239 19912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.707789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:37701 (local address 127.0.0.1:39054)
0809 16:11:14.708155 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.708165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.708216 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.708599 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.708600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.708602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:14.708658 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.708659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.709906 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.709907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.711056 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.711059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.711982 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.711983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.712040 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.712043 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.712087 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.712119 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":35,"thread_start_us":59,"threads_started":1}
I20260809 16:11:14.712479 19920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.707983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.16.224.131:37701)
0809 16:11:14.708379 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.708383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.708394 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:14.708432 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.708434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.708449 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.708535 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:14.708729 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.709822 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.711138 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.711841 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.711900 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.711947 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.712333 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.712377 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":89,"threads_started":1}
I20260809 16:11:14.712767 19550 tablet_service.cc:1467] Processing CreateTablet for tablet c03d9ed71e8f425f830817e14a67b61d (DEFAULT_TABLE table=test-table [id=55ed0f4f67094cd1b1aedf1fe3737894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:14.712883 19317 tablet_service.cc:1467] Processing CreateTablet for tablet c03d9ed71e8f425f830817e14a67b61d (DEFAULT_TABLE table=test-table [id=55ed0f4f67094cd1b1aedf1fe3737894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:14.713195 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03d9ed71e8f425f830817e14a67b61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:14.713209 19317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03d9ed71e8f425f830817e14a67b61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:14.713080 19774 tablet_service.cc:1467] Processing CreateTablet for tablet c03d9ed71e8f425f830817e14a67b61d (DEFAULT_TABLE table=test-table [id=55ed0f4f67094cd1b1aedf1fe3737894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:14.713402 19774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03d9ed71e8f425f830817e14a67b61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:14.716034 19922 tablet_bootstrap.cc:492] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Bootstrap starting.
I20260809 16:11:14.716696 19923 tablet_bootstrap.cc:492] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Bootstrap starting.
I20260809 16:11:14.716744 19922 tablet_bootstrap.cc:654] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:14.717099 19922 log.cc:824] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:14.717393 19923 tablet_bootstrap.cc:654] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:14.717701 19923 log.cc:824] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:14.717949 19922 tablet_bootstrap.cc:492] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: No bootstrap required, opened a new log
I20260809 16:11:14.718039 19922 ts_tablet_manager.cc:1397] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:14.718071 19924 tablet_bootstrap.cc:492] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Bootstrap starting.
I20260809 16:11:14.718456 19923 tablet_bootstrap.cc:492] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: No bootstrap required, opened a new log
I20260809 16:11:14.718518 19923 ts_tablet_manager.cc:1397] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:14.719113 19924 tablet_bootstrap.cc:654] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:14.719444 19924 log.cc:824] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:14.720212 19924 tablet_bootstrap.cc:492] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: No bootstrap required, opened a new log
I20260809 16:11:14.720099 19922 raft_consensus.cc:348] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.720263 19922 raft_consensus.cc:374] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:14.720281 19924 ts_tablet_manager.cc:1397] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:11:14.720299 19922 raft_consensus.cc:729] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d95a6dafcbb42ec973787eb7d1b1d6c, State: Initialized, Role: FOLLOWER
I20260809 16:11:14.720201 19923 raft_consensus.cc:348] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.720340 19923 raft_consensus.cc:374] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:14.720372 19923 raft_consensus.cc:729] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961876aca74343ff9116157257ffc885, State: Initialized, Role: FOLLOWER
I20260809 16:11:14.720471 19922 consensus_queue.cc:260] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.720513 19923 consensus_queue.cc:260] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.720737 19922 ts_tablet_manager.cc:1428] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:11:14.720770 19923 ts_tablet_manager.cc:1428] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:14.720842 19659 heartbeater.cc:503] Master 127.16.224.190:46631 was elected leader, sending a full tablet report...
I20260809 16:11:14.721091 19424 heartbeater.cc:503] Master 127.16.224.190:46631 was elected leader, sending a full tablet report...
I20260809 16:11:14.722062 19924 raft_consensus.cc:348] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.722196 19924 raft_consensus.cc:374] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:14.722218 19924 raft_consensus.cc:729] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82a9802236f4255808ef8bc27a7c8df, State: Initialized, Role: FOLLOWER
I20260809 16:11:14.722299 19924 consensus_queue.cc:260] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.722584 19924 ts_tablet_manager.cc:1428] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:11:14.722692 19886 heartbeater.cc:503] Master 127.16.224.190:46631 was elected leader, sending a full tablet report...
W20260809 16:11:14.759716 19660 tablet.cc:2369] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:14.839076 19932 raft_consensus.cc:482] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:14.839222 19932 raft_consensus.cc:504] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.839635 19932 leader_election.cc:288] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701), 961876aca74343ff9116157257ffc885 (127.16.224.130:46037)
I20260809 16:11:14.842433 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.839915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:32955 (local address 127.16.224.130:46037)
0809 16:11:14.839956 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.839958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.839969 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:14.840049 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.840050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.840057 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.840131 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:14.840295 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.840774 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.841714 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.842086 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.842148 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.842157 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.842305 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.842364 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:11:14.842433 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.839839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46037 (local address 127.16.224.129:32955)
0809 16:11:14.839906 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.839920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.839982 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.840147 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.840148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.840150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:14.840228 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.840230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.840865 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.840866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.841620 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.841622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.842159 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.842161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.842235 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.842238 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.842328 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.842364 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 16:11:14.842530 19920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.839897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:43987 (local address 127.16.224.131:37701)
0809 16:11:14.839956 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.839959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.840160 (+ 201us) server_negotiation.cc:407] Connection header received
0809 16:11:14.840243 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.840244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.840268 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.840326 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:14.840484 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.840956 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.841809 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.842169 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.842228 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.842236 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.842433 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.842473 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:11:14.842543 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.839841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:37701 (local address 127.16.224.129:43987)
0809 16:11:14.840103 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:14.840111 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:14.840183 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:14.840334 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:14.840336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:14.840338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:14.840421 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.840423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.841024 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.841025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.841730 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:14.841732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.842237 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.842238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:14.842340 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.842345 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:11:14.842419 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:11:14.842455 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":85,"thread_start_us":115,"threads_started":1}
I20260809 16:11:14.842813 19579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03d9ed71e8f425f830817e14a67b61d" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961876aca74343ff9116157257ffc885" is_pre_election: true
I20260809 16:11:14.842986 19579 raft_consensus.cc:2454] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 0.
I20260809 16:11:14.843014 19815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03d9ed71e8f425f830817e14a67b61d" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82a9802236f4255808ef8bc27a7c8df" is_pre_election: true
I20260809 16:11:14.843174 19815 raft_consensus.cc:2454] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 0.
I20260809 16:11:14.843211 19249 leader_election.cc:302] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d95a6dafcbb42ec973787eb7d1b1d6c, 961876aca74343ff9116157257ffc885; no voters:
I20260809 16:11:14.843333 19932 raft_consensus.cc:2781] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:14.843390 19932 raft_consensus.cc:482] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:14.843417 19932 raft_consensus.cc:3037] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:14.844110 19932 raft_consensus.cc:504] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.844231 19932 leader_election.cc:288] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 election: Requested vote from peers e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701), 961876aca74343ff9116157257ffc885 (127.16.224.130:46037)
I20260809 16:11:14.844388 19579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03d9ed71e8f425f830817e14a67b61d" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961876aca74343ff9116157257ffc885"
I20260809 16:11:14.844415 19815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03d9ed71e8f425f830817e14a67b61d" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82a9802236f4255808ef8bc27a7c8df"
I20260809 16:11:14.844472 19579 raft_consensus.cc:3037] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:14.844481 19815 raft_consensus.cc:3037] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:14.845208 19579 raft_consensus.cc:2454] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 1.
I20260809 16:11:14.845230 19815 raft_consensus.cc:2454] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 1.
I20260809 16:11:14.845367 19249 leader_election.cc:302] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d95a6dafcbb42ec973787eb7d1b1d6c, 961876aca74343ff9116157257ffc885; no voters:
I20260809 16:11:14.845523 19932 raft_consensus.cc:2781] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:14.845651 19932 raft_consensus.cc:686] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 LEADER]: Becoming Leader. State: Replica: 2d95a6dafcbb42ec973787eb7d1b1d6c, State: Running, Role: LEADER
I20260809 16:11:14.845741 19932 consensus_queue.cc:237] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } }
I20260809 16:11:14.846624 19096 catalog_manager.cc:5495] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d95a6dafcbb42ec973787eb7d1b1d6c (127.16.224.129). New cstate: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } health_report { overall_health: HEALTHY } } }
I20260809 16:11:14.861212 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:14.858768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.16.224.129:34181)
0809 16:11:14.858836 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:14.858838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:14.858846 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:14.858943 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:14.858944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:14.858963 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:14.859018 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:14.859150 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.859636 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.860421 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:14.860612 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:14.860666 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:14.860874 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:14.860968 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:14.861034 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:14.861088 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 16:11:14.861110 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:11:14.863405 19815 raft_consensus.cc:1261] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Refusing update from remote peer 2d95a6dafcbb42ec973787eb7d1b1d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:14.863405 19579 raft_consensus.cc:1261] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Refusing update from remote peer 2d95a6dafcbb42ec973787eb7d1b1d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:14.863777 19932 consensus_queue.cc:1035] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:14.865605 19932 consensus_queue.cc:1035] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:14.870371 19097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:14.870527 19097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:14.871425 19097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60: Sending DeleteTablet for 3 replicas of tablet c03d9ed71e8f425f830817e14a67b61d
I20260809 16:11:14.871816 19774 tablet_service.cc:1514] Processing DeleteTablet for tablet c03d9ed71e8f425f830817e14a67b61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:14 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:39054
I20260809 16:11:14.871851 19317 tablet_service.cc:1514] Processing DeleteTablet for tablet c03d9ed71e8f425f830817e14a67b61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:14 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:33922
I20260809 16:11:14.871857 19550 tablet_service.cc:1514] Processing DeleteTablet for tablet c03d9ed71e8f425f830817e14a67b61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:14 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:46786
I20260809 16:11:14.872071 19964 tablet_replica.cc:330] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: stopping tablet replica
I20260809 16:11:14.872184 19957 tablet_replica.cc:330] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: stopping tablet replica
I20260809 16:11:14.872207 19964 raft_consensus.cc:2229] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:14.872277 19957 raft_consensus.cc:2229] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:14.872269 19097 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:14.872409 19097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:14.872421 19956 tablet_replica.cc:330] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: stopping tablet replica
I20260809 16:11:14.872494 19956 raft_consensus.cc:2229] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:14.872586 19956 raft_consensus.cc:2258] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:14.872605 19964 raft_consensus.cc:2258] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:14.872725 19957 raft_consensus.cc:2258] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:14.873231 19956 ts_tablet_manager.cc:1905] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:14.873848 19317 tablet_service.cc:1467] Processing CreateTablet for tablet 0810f8b0387f428a861c81f9cadc3255 (DEFAULT_TABLE table=test-table [id=d7707ac79acd4c8daf5384025b5690b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:14.873888 19774 tablet_service.cc:1467] Processing CreateTablet for tablet 0810f8b0387f428a861c81f9cadc3255 (DEFAULT_TABLE table=test-table [id=d7707ac79acd4c8daf5384025b5690b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:14.873878 19550 tablet_service.cc:1467] Processing CreateTablet for tablet 0810f8b0387f428a861c81f9cadc3255 (DEFAULT_TABLE table=test-table [id=d7707ac79acd4c8daf5384025b5690b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:14.873956 19317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0810f8b0387f428a861c81f9cadc3255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:14.873970 19774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0810f8b0387f428a861c81f9cadc3255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:14.873997 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0810f8b0387f428a861c81f9cadc3255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:14.875356 19923 tablet_bootstrap.cc:492] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: Bootstrap starting.
I20260809 16:11:14.875392 19922 tablet_bootstrap.cc:492] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Bootstrap starting.
I20260809 16:11:14.875617 19924 tablet_bootstrap.cc:492] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Bootstrap starting.
I20260809 16:11:14.875798 19922 tablet_bootstrap.cc:654] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:14.875840 19923 tablet_bootstrap.cc:654] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:14.876000 19924 tablet_bootstrap.cc:654] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:14.876653 19964 ts_tablet_manager.cc:1905] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:14.876823 19957 ts_tablet_manager.cc:1905] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:14.877802 19956 ts_tablet_manager.cc:1918] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:14.878135 19964 ts_tablet_manager.cc:1918] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:14.878190 19956 log.cc:1196] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/c03d9ed71e8f425f830817e14a67b61d
I20260809 16:11:14.878213 19964 log.cc:1196] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/c03d9ed71e8f425f830817e14a67b61d
I20260809 16:11:14.878392 19957 ts_tablet_manager.cc:1918] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:14.878494 19957 log.cc:1196] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/c03d9ed71e8f425f830817e14a67b61d
I20260809 16:11:14.878530 19964 ts_tablet_manager.cc:1939] T c03d9ed71e8f425f830817e14a67b61d P 961876aca74343ff9116157257ffc885: Deleting consensus metadata
I20260809 16:11:14.878577 19956 ts_tablet_manager.cc:1939] T c03d9ed71e8f425f830817e14a67b61d P 2d95a6dafcbb42ec973787eb7d1b1d6c: Deleting consensus metadata
I20260809 16:11:14.878824 19957 ts_tablet_manager.cc:1939] T c03d9ed71e8f425f830817e14a67b61d P e82a9802236f4255808ef8bc27a7c8df: Deleting consensus metadata
I20260809 16:11:14.878927 19076 catalog_manager.cc:4841] TS 2d95a6dafcbb42ec973787eb7d1b1d6c (127.16.224.129:34181): tablet c03d9ed71e8f425f830817e14a67b61d (table test-table [id=55ed0f4f67094cd1b1aedf1fe3737894]) successfully deleted
I20260809 16:11:14.879069 19076 catalog_manager.cc:4841] TS 961876aca74343ff9116157257ffc885 (127.16.224.130:46037): tablet c03d9ed71e8f425f830817e14a67b61d (table test-table [id=55ed0f4f67094cd1b1aedf1fe3737894]) successfully deleted
I20260809 16:11:14.879123 19922 tablet_bootstrap.cc:492] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: No bootstrap required, opened a new log
I20260809 16:11:14.879174 19922 ts_tablet_manager.cc:1397] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 16:11:14.879329 19924 tablet_bootstrap.cc:492] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: No bootstrap required, opened a new log
I20260809 16:11:14.879298 19922 raft_consensus.cc:348] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:14.879369 19922 raft_consensus.cc:374] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:14.879375 19924 ts_tablet_manager.cc:1397] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 16:11:14.879390 19922 raft_consensus.cc:729] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d95a6dafcbb42ec973787eb7d1b1d6c, State: Initialized, Role: FOLLOWER
I20260809 16:11:14.879429 19922 consensus_queue.cc:260] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:14.879433 19073 catalog_manager.cc:4841] TS e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701): tablet c03d9ed71e8f425f830817e14a67b61d (table test-table [id=55ed0f4f67094cd1b1aedf1fe3737894]) successfully deleted
I20260809 16:11:14.879508 19922 ts_tablet_manager.cc:1428] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:14.879522 19924 raft_consensus.cc:348] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:14.879582 19924 raft_consensus.cc:374] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:14.879601 19924 raft_consensus.cc:729] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82a9802236f4255808ef8bc27a7c8df, State: Initialized, Role: FOLLOWER
I20260809 16:11:14.879644 19924 consensus_queue.cc:260] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:14.879729 19924 ts_tablet_manager.cc:1428] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:14.880359 19923 tablet_bootstrap.cc:492] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: No bootstrap required, opened a new log
I20260809 16:11:14.880414 19923 ts_tablet_manager.cc:1397] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 16:11:14.880558 19923 raft_consensus.cc:348] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:14.880627 19923 raft_consensus.cc:374] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:14.880656 19923 raft_consensus.cc:729] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961876aca74343ff9116157257ffc885, State: Initialized, Role: FOLLOWER
I20260809 16:11:14.880702 19923 consensus_queue.cc:260] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:14.880787 19923 ts_tablet_manager.cc:1428] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260809 16:11:14.916366 19888 tablet.cc:2369] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:11:15.118824 19427 tablet.cc:2369] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:15.203994 19933 raft_consensus.cc:482] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:15.204108 19933 raft_consensus.cc:504] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.204490 19933 leader_election.cc:288] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d95a6dafcbb42ec973787eb7d1b1d6c (127.16.224.129:34181), e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701)
I20260809 16:11:15.207356 19920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.204817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:55579 (local address 127.16.224.131:37701)
0809 16:11:15.204867 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.204869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.204884 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:11:15.204968 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.204969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.204976 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.205029 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:15.205158 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.205648 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.206677 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.207040 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.207099 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.207109 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.207241 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.207283 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:11:15.207370 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.204666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:34181 (local address 127.16.224.130:35647)
0809 16:11:15.204802 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:15.204813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:15.204877 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:15.205025 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:15.205026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:15.205029 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:15.205109 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.205111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.205726 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.205727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.206591 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.206594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.207132 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.207133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.207200 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.207203 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:15.207254 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:11:15.207292 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":42,"thread_start_us":71,"threads_started":1}
I20260809 16:11:15.207376 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.204747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:35647 (local address 127.16.224.129:34181)
0809 16:11:15.204804 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.204807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.204870 (+ 63us) server_negotiation.cc:407] Connection header received
0809 16:11:15.204944 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.204945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.204959 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.205004 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:15.205175 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.205642 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.206676 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.207047 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.207111 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.207119 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.207269 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.207319 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:11:15.207370 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.204670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:37701 (local address 127.16.224.130:55579)
0809 16:11:15.204825 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:15.204837 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:15.204898 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:15.205039 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:15.205040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:15.205043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:15.205099 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.205100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.205732 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.205733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.206590 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.206593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.207111 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.207112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.207178 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.207182 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:15.207223 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:11:15.207262 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":20,"thread_start_us":65,"threads_started":1}
I20260809 16:11:15.207712 19352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0810f8b0387f428a861c81f9cadc3255" candidate_uuid: "961876aca74343ff9116157257ffc885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" is_pre_election: true
I20260809 16:11:15.207783 19815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0810f8b0387f428a861c81f9cadc3255" candidate_uuid: "961876aca74343ff9116157257ffc885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82a9802236f4255808ef8bc27a7c8df" is_pre_election: true
I20260809 16:11:15.207866 19815 raft_consensus.cc:2454] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961876aca74343ff9116157257ffc885 in term 0.
I20260809 16:11:15.207867 19352 raft_consensus.cc:2454] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961876aca74343ff9116157257ffc885 in term 0.
I20260809 16:11:15.208074 19464 leader_election.cc:302] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961876aca74343ff9116157257ffc885, e82a9802236f4255808ef8bc27a7c8df; no voters:
I20260809 16:11:15.208220 19933 raft_consensus.cc:2781] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:15.208262 19933 raft_consensus.cc:482] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:15.208288 19933 raft_consensus.cc:3037] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:15.208984 19933 raft_consensus.cc:504] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.209111 19933 leader_election.cc:288] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [CANDIDATE]: Term 1 election: Requested vote from peers 2d95a6dafcbb42ec973787eb7d1b1d6c (127.16.224.129:34181), e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701)
I20260809 16:11:15.209299 19815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0810f8b0387f428a861c81f9cadc3255" candidate_uuid: "961876aca74343ff9116157257ffc885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82a9802236f4255808ef8bc27a7c8df"
I20260809 16:11:15.209295 19352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0810f8b0387f428a861c81f9cadc3255" candidate_uuid: "961876aca74343ff9116157257ffc885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c"
I20260809 16:11:15.209381 19815 raft_consensus.cc:3037] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:15.209396 19352 raft_consensus.cc:3037] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:15.209959 19815 raft_consensus.cc:2454] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961876aca74343ff9116157257ffc885 in term 1.
I20260809 16:11:15.210103 19352 raft_consensus.cc:2454] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961876aca74343ff9116157257ffc885 in term 1.
I20260809 16:11:15.210124 19464 leader_election.cc:302] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961876aca74343ff9116157257ffc885, e82a9802236f4255808ef8bc27a7c8df; no voters:
I20260809 16:11:15.210284 19933 raft_consensus.cc:2781] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:15.210464 19933 raft_consensus.cc:686] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 1 LEADER]: Becoming Leader. State: Replica: 961876aca74343ff9116157257ffc885, State: Running, Role: LEADER
I20260809 16:11:15.210526 19933 consensus_queue.cc:237] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.211115 19097 catalog_manager.cc:5495] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 reported cstate change: term changed from 0 to 1, leader changed from <none> to 961876aca74343ff9116157257ffc885 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "961876aca74343ff9116157257ffc885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } health_report { overall_health: UNKNOWN } } }
Aug 09 16:11:15 dist-test-slave-fgb3 krb5kdc[19022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:15.253487 19990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.247081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58032 (local address 127.16.224.190:46631)
0809 16:11:15.247233 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.247235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.247247 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:11:15.247316 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.247317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.247327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.247388 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:15.247822 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.248325 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.248970 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.249149 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.249192 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.251193 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:15.251194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:15.251194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:15.251200 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:15.252414 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:15.252835 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:15.252836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:15.252836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:15.252887 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:15.253040 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:15.253040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:15.253041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:15.253142 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:15.253189 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.253338 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.253412 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260809 16:11:15.253593 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.247021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46631 (local address 127.0.0.1:58032)
0809 16:11:15.247186 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:15.247199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:15.247263 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:15.247403 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:15.247404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:15.247712 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:15.247767 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.247769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.248380 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.248380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.248918 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.248919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.249231 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.249233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.249285 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.249750 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:15.249754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:15.251112 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:15.252529 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:15.252540 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:15.252543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:15.252761 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:15.252934 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:15.252935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:15.252935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:15.252989 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:15.253214 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:15.253220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:15.253262 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 16:11:15.253317 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:11:15.253476 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":61,"thread_start_us":90,"threads_started":1}
I20260809 16:11:15.255748 19097 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:58032:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:11:15.256031 19097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:15.257602 19774 tablet_service.cc:1467] Processing CreateTablet for tablet b27d8bc5f38a47f498cfaf189fda4569 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2f645eaeedde490bbd1b2062281bc966]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:15.257618 19317 tablet_service.cc:1467] Processing CreateTablet for tablet b27d8bc5f38a47f498cfaf189fda4569 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2f645eaeedde490bbd1b2062281bc966]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:15.257799 19317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27d8bc5f38a47f498cfaf189fda4569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:15.257808 19774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27d8bc5f38a47f498cfaf189fda4569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:15.259019 19550 tablet_service.cc:1467] Processing CreateTablet for tablet b27d8bc5f38a47f498cfaf189fda4569 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2f645eaeedde490bbd1b2062281bc966]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:15.259171 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27d8bc5f38a47f498cfaf189fda4569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:15.259963 19922 tablet_bootstrap.cc:492] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Bootstrap starting.
I20260809 16:11:15.260313 19922 tablet_bootstrap.cc:654] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:15.260412 19923 tablet_bootstrap.cc:492] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: Bootstrap starting.
I20260809 16:11:15.260573 19924 tablet_bootstrap.cc:492] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: Bootstrap starting.
I20260809 16:11:15.260818 19923 tablet_bootstrap.cc:654] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:15.260931 19922 tablet_bootstrap.cc:492] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: No bootstrap required, opened a new log
I20260809 16:11:15.260947 19924 tablet_bootstrap.cc:654] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:15.260977 19922 ts_tablet_manager.cc:1397] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:15.261103 19922 raft_consensus.cc:348] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.261176 19922 raft_consensus.cc:374] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:15.261194 19922 raft_consensus.cc:729] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d95a6dafcbb42ec973787eb7d1b1d6c, State: Initialized, Role: FOLLOWER
I20260809 16:11:15.261242 19922 consensus_queue.cc:260] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.261344 19922 ts_tablet_manager.cc:1428] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:15.261449 19940 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.261508 19923 tablet_bootstrap.cc:492] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: No bootstrap required, opened a new log
I20260809 16:11:15.261523 19940 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:15.261551 19923 ts_tablet_manager.cc:1397] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:15.261651 19924 tablet_bootstrap.cc:492] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: No bootstrap required, opened a new log
I20260809 16:11:15.261691 19924 ts_tablet_manager.cc:1397] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:15.261693 19923 raft_consensus.cc:348] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.261747 19923 raft_consensus.cc:374] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:15.261759 19923 raft_consensus.cc:729] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961876aca74343ff9116157257ffc885, State: Initialized, Role: FOLLOWER
I20260809 16:11:15.261801 19923 consensus_queue.cc:260] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.261828 19924 raft_consensus.cc:348] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.261880 19923 ts_tablet_manager.cc:1428] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:15.261895 19924 raft_consensus.cc:374] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:15.261914 19924 raft_consensus.cc:729] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82a9802236f4255808ef8bc27a7c8df, State: Initialized, Role: FOLLOWER
I20260809 16:11:15.261953 19924 consensus_queue.cc:260] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.261935 19933 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.262012 19933 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:15.262051 19924 ts_tablet_manager.cc:1428] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:15.262156 19938 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.262260 19938 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:15.290663 19940 raft_consensus.cc:482] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:15.290763 19940 raft_consensus.cc:504] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.290923 19940 leader_election.cc:288] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 961876aca74343ff9116157257ffc885 (127.16.224.130:46037), e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701)
I20260809 16:11:15.291145 19579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27d8bc5f38a47f498cfaf189fda4569" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961876aca74343ff9116157257ffc885" is_pre_election: true
I20260809 16:11:15.291147 19815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27d8bc5f38a47f498cfaf189fda4569" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82a9802236f4255808ef8bc27a7c8df" is_pre_election: true
I20260809 16:11:15.291255 19579 raft_consensus.cc:2454] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 0.
I20260809 16:11:15.291258 19815 raft_consensus.cc:2454] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 0.
I20260809 16:11:15.291409 19249 leader_election.cc:302] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d95a6dafcbb42ec973787eb7d1b1d6c, 961876aca74343ff9116157257ffc885; no voters:
I20260809 16:11:15.291538 19940 raft_consensus.cc:2781] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:15.291601 19940 raft_consensus.cc:482] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:15.291617 19940 raft_consensus.cc:3037] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:15.292366 19940 raft_consensus.cc:504] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.292496 19940 leader_election.cc:288] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 election: Requested vote from peers 961876aca74343ff9116157257ffc885 (127.16.224.130:46037), e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701)
I20260809 16:11:15.292680 19815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27d8bc5f38a47f498cfaf189fda4569" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82a9802236f4255808ef8bc27a7c8df"
I20260809 16:11:15.292719 19579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27d8bc5f38a47f498cfaf189fda4569" candidate_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961876aca74343ff9116157257ffc885"
I20260809 16:11:15.292757 19815 raft_consensus.cc:3037] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:15.292788 19579 raft_consensus.cc:3037] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:15.293512 19815 raft_consensus.cc:2454] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 1.
I20260809 16:11:15.293548 19579 raft_consensus.cc:2454] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d95a6dafcbb42ec973787eb7d1b1d6c in term 1.
I20260809 16:11:15.293691 19243 leader_election.cc:302] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d95a6dafcbb42ec973787eb7d1b1d6c, e82a9802236f4255808ef8bc27a7c8df; no voters:
I20260809 16:11:15.293793 19940 raft_consensus.cc:2781] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:15.293864 19940 raft_consensus.cc:686] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 LEADER]: Becoming Leader. State: Replica: 2d95a6dafcbb42ec973787eb7d1b1d6c, State: Running, Role: LEADER
I20260809 16:11:15.293917 19940 consensus_queue.cc:237] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } }
I20260809 16:11:15.294170 19932 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: TxnStatusTablet state changed. Reason: New leader 2d95a6dafcbb42ec973787eb7d1b1d6c. Latest consensus state: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.294239 19932 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:15.294450 19997 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:11:15.294581 19997 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:11:15.294500 19096 catalog_manager.cc:5495] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d95a6dafcbb42ec973787eb7d1b1d6c (127.16.224.129). New cstate: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:15.307914 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.305389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.16.224.129:34181)
0809 16:11:15.305431 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.305433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.305461 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:11:15.305549 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.305550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.305559 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.305604 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:15.305726 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.306197 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.307305 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.307487 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.307532 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.307700 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:15.307726 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:15.307757 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.307812 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.307827 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:11:15.307947 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.305322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:34181 (local address 127.0.0.1:33926)
0809 16:11:15.305415 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:15.305427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:15.305479 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:15.305612 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:15.305613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:15.305614 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:15.305673 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.305675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.306263 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.306264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.307229 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.307233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.307547 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.307548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.307610 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.307631 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:15.307758 (+ 127us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:15.307764 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:11:15.307811 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:11:15.307866 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 16:11:15.309814 19579 raft_consensus.cc:1261] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Refusing update from remote peer 2d95a6dafcbb42ec973787eb7d1b1d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:15.309810 19815 raft_consensus.cc:1261] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Refusing update from remote peer 2d95a6dafcbb42ec973787eb7d1b1d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:15.310039 19932 consensus_queue.cc:1035] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:15.310159 19940 consensus_queue.cc:1035] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:15.310684 19938 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: TxnStatusTablet state changed. Reason: New leader 2d95a6dafcbb42ec973787eb7d1b1d6c. Latest consensus state: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.310786 19938 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:15.310771 19933 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: TxnStatusTablet state changed. Reason: New leader 2d95a6dafcbb42ec973787eb7d1b1d6c. Latest consensus state: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.310868 19933 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:15.311471 19940 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.311559 19940 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:15.311645 19940 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.311697 19940 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P 2d95a6dafcbb42ec973787eb7d1b1d6c: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:15.311861 19933 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.311929 19933 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P 961876aca74343ff9116157257ffc885: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:15.312074 19938 tablet_replica.cc:439] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 } } peers { permanent_uuid: "961876aca74343ff9116157257ffc885" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46037 } } peers { permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 } } }
I20260809 16:11:15.312151 19938 tablet_replica.cc:442] T b27d8bc5f38a47f498cfaf189fda4569 P e82a9802236f4255808ef8bc27a7c8df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:15.312116 19945 mvcc.cc:205] Tried to move back new op lower bound from 7316651521267310592 to 7316651521204215808. Current Snapshot: MvccSnapshot[applied={T|T < 7316651521267310592 or (T in {7316651521267310592})}]
I20260809 16:11:15.314921 19949 mvcc.cc:205] Tried to move back new op lower bound from 7316651521267310592 to 7316651521204215808. Current Snapshot: MvccSnapshot[applied={T|T < 7316651521267310592 or (T in {7316651521267310592})}]
I20260809 16:11:15.321364 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.318231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.16.224.130:46037)
0809 16:11:15.318268 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.318270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.318277 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:15.318353 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.318354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.318378 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.318448 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:15.318619 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.319335 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.320334 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.320632 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.320687 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.320881 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:15.321022 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:15.321089 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.321261 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.321286 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:11:15.335321 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.332833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:34181 (local address 127.16.224.130:59853)
0809 16:11:15.333038 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:15.333047 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:15.333105 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:15.333226 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:15.333226 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:15.333228 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:15.333291 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.333293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.333886 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.333887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.334552 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.334555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.334886 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.334886 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.334953 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.334969 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:15.335141 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:15.335144 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:15.335189 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:11:15.335231 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260809 16:11:15.335296 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.332906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:59853 (local address 127.16.224.129:34181)
0809 16:11:15.332962 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.332964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.333104 (+ 140us) server_negotiation.cc:407] Connection header received
0809 16:11:15.333157 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.333159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.333170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.333222 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:15.333355 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.333816 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.334625 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.334826 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.334880 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.335038 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:15.335075 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:15.335139 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.335194 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.335215 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:11:15.337874 19352 raft_consensus.cc:1261] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Refusing update from remote peer 961876aca74343ff9116157257ffc885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:15.338120 19815 raft_consensus.cc:1261] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Refusing update from remote peer 961876aca74343ff9116157257ffc885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:15.338148 19933 consensus_queue.cc:1035] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d95a6dafcbb42ec973787eb7d1b1d6c" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 34181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:15.338363 19933 consensus_queue.cc:1035] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82a9802236f4255808ef8bc27a7c8df" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:15.346297 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.343845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:52447 (local address 127.16.224.130:46037)
0809 16:11:15.343883 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.343884 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.344013 (+ 129us) server_negotiation.cc:407] Connection header received
0809 16:11:15.344056 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.344057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.344069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.344111 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:15.344231 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.344734 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.345603 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.345787 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.345846 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.346016 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:15.346055 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:15.346110 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.346202 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.346223 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:11:15.346287 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.343798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46037 (local address 127.16.224.129:52447)
0809 16:11:15.343950 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:15.343960 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:15.344011 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:15.344112 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:15.344113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:15.344115 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:15.344177 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.344179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.344805 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.344806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.345543 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:15.345547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.345847 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.345847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:15.345912 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.345931 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:15.346115 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:15.346116 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:11:15.346154 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 16:11:15.346203 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260809 16:11:15.347462 19943 mvcc.cc:205] Tried to move back new op lower bound from 7316651521381912576 to 7316651520862793728. Current Snapshot: MvccSnapshot[applied={T|T < 7316651521381912576}]
I20260809 16:11:15.354068 19097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:15.354174 19097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:15.354816 19097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e552c268f25847f3903b91cce452ea60: Sending DeleteTablet for 3 replicas of tablet 0810f8b0387f428a861c81f9cadc3255
I20260809 16:11:15.355044 19317 tablet_service.cc:1514] Processing DeleteTablet for tablet 0810f8b0387f428a861c81f9cadc3255 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:15 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:33922
I20260809 16:11:15.355063 19550 tablet_service.cc:1514] Processing DeleteTablet for tablet 0810f8b0387f428a861c81f9cadc3255 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:15 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:46786
I20260809 16:11:15.355135 19956 tablet_replica.cc:330] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: stopping tablet replica
I20260809 16:11:15.355152 19964 tablet_replica.cc:330] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: stopping tablet replica
I20260809 16:11:15.355182 19956 raft_consensus.cc:2229] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:15.355216 19956 raft_consensus.cc:2258] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:15.355223 19964 raft_consensus.cc:2229] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:15.355304 19964 raft_consensus.cc:2258] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:15.355288 19774 tablet_service.cc:1514] Processing DeleteTablet for tablet 0810f8b0387f428a861c81f9cadc3255 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:15 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:39054
I20260809 16:11:15.355406 19957 tablet_replica.cc:330] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: stopping tablet replica
I20260809 16:11:15.355451 19957 raft_consensus.cc:2229] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:15.355489 19957 raft_consensus.cc:2258] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:15.355609 19956 ts_tablet_manager.cc:1905] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:15.355620 19964 ts_tablet_manager.cc:1905] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:15.357133 19956 ts_tablet_manager.cc:1918] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:15.357187 19956 log.cc:1196] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/0810f8b0387f428a861c81f9cadc3255
I20260809 16:11:15.357231 19964 ts_tablet_manager.cc:1918] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:15.357295 19964 log.cc:1196] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/0810f8b0387f428a861c81f9cadc3255
I20260809 16:11:15.357401 19956 ts_tablet_manager.cc:1939] T 0810f8b0387f428a861c81f9cadc3255 P 2d95a6dafcbb42ec973787eb7d1b1d6c: Deleting consensus metadata
I20260809 16:11:15.357542 19964 ts_tablet_manager.cc:1939] T 0810f8b0387f428a861c81f9cadc3255 P 961876aca74343ff9116157257ffc885: Deleting consensus metadata
I20260809 16:11:15.357689 19076 catalog_manager.cc:4841] TS 2d95a6dafcbb42ec973787eb7d1b1d6c (127.16.224.129:34181): tablet 0810f8b0387f428a861c81f9cadc3255 (table test-table [id=d7707ac79acd4c8daf5384025b5690b5]) successfully deleted
I20260809 16:11:15.357859 19076 catalog_manager.cc:4841] TS 961876aca74343ff9116157257ffc885 (127.16.224.130:46037): tablet 0810f8b0387f428a861c81f9cadc3255 (table test-table [id=d7707ac79acd4c8daf5384025b5690b5]) successfully deleted
I20260809 16:11:15.358059 19957 ts_tablet_manager.cc:1905] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:15.359197 19957 ts_tablet_manager.cc:1918] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:15.359249 19957 log.cc:1196] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/0810f8b0387f428a861c81f9cadc3255
I20260809 16:11:15.359438 19957 ts_tablet_manager.cc:1939] T 0810f8b0387f428a861c81f9cadc3255 P e82a9802236f4255808ef8bc27a7c8df: Deleting consensus metadata
I20260809 16:11:15.359719 19073 catalog_manager.cc:4841] TS e82a9802236f4255808ef8bc27a7c8df (127.16.224.131:37701): tablet 0810f8b0387f428a861c81f9cadc3255 (table test-table [id=d7707ac79acd4c8daf5384025b5690b5]) successfully deleted
Aug 09 16:11:15 dist-test-slave-fgb3 krb5kdc[19022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291874, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:15.373883 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.362600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.16.224.129:34181)
0809 16:11:15.362646 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.362648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.362881 (+ 233us) server_negotiation.cc:407] Connection header received
0809 16:11:15.362941 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.362942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.362954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.363000 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:15.363544 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.364020 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.364724 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.364940 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.364996 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.370977 (+ 5981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:15.370988 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:15.370988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:15.370997 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:15.372729 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:15.373109 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:15.373113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:15.373114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:15.373176 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:15.373353 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:15.373353 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:15.373354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:15.373504 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:15.373637 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.373706 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.373791 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:11:15.374228 19417 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33932
I20260809 16:11:15.380903 19990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:15.375249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.16.224.190:46631)
0809 16:11:15.375301 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:15.375303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:15.375468 (+ 165us) server_negotiation.cc:407] Connection header received
0809 16:11:15.375528 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:15.375530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:15.375542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:15.375586 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:15.376063 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.376567 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.377224 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:15.377441 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:15.377495 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:15.378624 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:15.378626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:15.378626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:15.378634 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:15.379883 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:15.380176 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:15.380177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:15.380177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:15.380233 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:15.380398 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:15.380399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:15.380399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:15.380511 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:15.380599 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:15.380707 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:11:15.380793 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:11:15.381191 19097 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58048
I20260809 16:11:15.381830 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19215
I20260809 16:11:15.400588 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19438
I20260809 16:11:15.412798 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19670
I20260809 16:11:15.422605 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19043
2026-08-09T16:11:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2684 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:15 dist-test-slave-fgb3 krb5kdc[20037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:15 dist-test-slave-fgb3 krb5kdc[20037](info): set up 2 sockets
Aug 09 16:11:15 dist-test-slave-fgb3 krb5kdc[20037](info): commencing operation
krb5kdc: starting...
W20260809 16:11:16.645777 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 16:11:16.825263 17282 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.179s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 16:11:16.992185 17282 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.167s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 16:11:17.125468 17282 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.133s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/test-user.keytab.
W20260809 16:11:17.442587 17282 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.317s user 0.000s sys 0.002s
Aug 09 16:11:17 dist-test-slave-fgb3 krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:17.483078 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43391
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:43391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:17.565132 20054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:17.565332 20054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:17.565353 20054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:17.566442 20054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:17.566473 20054 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:11:17.566485 20054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:17.566496 20054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:17.566507 20054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:17.566517 20054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:17.567945 20054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:43391
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:17.568140 20054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:17.568315 20054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:17.572918 20066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:17.572975 20063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:17.572981 20054 server_base.cc:1034] running on GCE node
W20260809 16:11:17.572929 20061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:17.573472 20054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:17.573750 20054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:17.574890 20054 hybrid_clock.cc:648] HybridClock initialized: now 1786291877574871 us; error 35 us; skew 500 ppm
Aug 09 16:11:17 dist-test-slave-fgb3 krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:17.577255 20054 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:17.577662 20054 webserver.cc:460] Webserver started at http://127.16.224.190:32947/ using document root <none> and password file <none>
I20260809 16:11:17.577863 20054 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:17.577911 20054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:17.578049 20054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:17.579141 20054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "8d5962fb43e549f2a7a3b878eee5d849"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "ffad22b0e6f0114723e98078542a0b72"
server_key_iv: "f3799a8070b7aeb6169a85e2fa9f31dc"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.579483 20054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "8d5962fb43e549f2a7a3b878eee5d849"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "ffad22b0e6f0114723e98078542a0b72"
server_key_iv: "f3799a8070b7aeb6169a85e2fa9f31dc"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.580716 20054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:17.581346 20075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:17.581506 20054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:17.581591 20054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "8d5962fb43e549f2a7a3b878eee5d849"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "ffad22b0e6f0114723e98078542a0b72"
server_key_iv: "f3799a8070b7aeb6169a85e2fa9f31dc"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.581652 20054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:17.604595 20054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:17.606966 20054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:17.607118 20054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:17.611867 20054 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:43391
I20260809 16:11:17.611917 20184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:43391 every 8 connection(s)
I20260809 16:11:17.612258 20054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:17.612977 20185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:17.615458 20185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Bootstrap starting.
I20260809 16:11:17.616082 20185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:17.616350 20185 log.cc:824] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:17.617107 20185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: No bootstrap required, opened a new log
I20260809 16:11:17.617769 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20054
I20260809 16:11:17.617853 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:17.618037 17282 external_mini_cluster.cc:1428] Setting key d587089accda3b6d09c3aa527e002158
I20260809 16:11:17.619314 20185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5962fb43e549f2a7a3b878eee5d849" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43391 } }
I20260809 16:11:17.619472 20185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:17.619496 20185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d5962fb43e549f2a7a3b878eee5d849, State: Initialized, Role: FOLLOWER
I20260809 16:11:17.619611 20185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5962fb43e549f2a7a3b878eee5d849" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43391 } }
I20260809 16:11:17.619689 20185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:17.619731 20185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:17.619781 20185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:17.620353 20185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5962fb43e549f2a7a3b878eee5d849" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43391 } }
I20260809 16:11:17.620456 20185 leader_election.cc:302] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d5962fb43e549f2a7a3b878eee5d849; no voters:
I20260809 16:11:17.620622 20185 leader_election.cc:288] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:17.620664 20195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:17.620781 20195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [term 1 LEADER]: Becoming Leader. State: Replica: 8d5962fb43e549f2a7a3b878eee5d849, State: Running, Role: LEADER
I20260809 16:11:17.620910 20185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:17.620891 20195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5962fb43e549f2a7a3b878eee5d849" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43391 } }
I20260809 16:11:17.621227 20195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d5962fb43e549f2a7a3b878eee5d849. Latest consensus state: current_term: 1 leader_uuid: "8d5962fb43e549f2a7a3b878eee5d849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5962fb43e549f2a7a3b878eee5d849" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43391 } } }
I20260809 16:11:17.621309 20195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:17.621237 20198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d5962fb43e549f2a7a3b878eee5d849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5962fb43e549f2a7a3b878eee5d849" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43391 } } }
I20260809 16:11:17.621460 20198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:17.621538 20200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:17.622076 20200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:17.624167 20200 catalog_manager.cc:1347] Generated new cluster ID: 41898ad92a8241bda158b2a8b672054d
I20260809 16:11:17.624218 20200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:11:17 dist-test-slave-fgb3 krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:17.630349 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:17.618686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.16.224.190:43391)
0809 16:11:17.618879 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:17.618888 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:17.618916 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:11:17.619390 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:17.619397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:17.619449 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:17.619599 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:17.620246 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.621512 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.622668 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.623295 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.623405 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:17.626033 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:17.626037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:17.626037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:17.626051 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:17.628671 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:17.629041 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:17.629044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:17.629045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:17.629167 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:17.629343 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:17.629344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:17.629344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:17.629667 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:17.629838 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:17.629923 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:11:17.630071 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260809 16:11:17.658604 20200 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:17.659123 20200 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:17.671301 20200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Generated new TSK 0
I20260809 16:11:17.671494 20200 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:17.700434 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43391
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:17.804589 20231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:17.804807 20231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:17.804832 20231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:17.805915 20231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:17.805949 20231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:17.805967 20231 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:11:17.805984 20231 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:17.805999 20231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:17.807451 20231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:43391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:17.807641 20231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:17.807824 20231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:17.812765 20241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:17.812879 20231 server_base.cc:1034] running on GCE node
W20260809 16:11:17.812747 20244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:17.812775 20240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:17.813206 20231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:17.813453 20231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:17.814599 20231 hybrid_clock.cc:648] HybridClock initialized: now 1786291877814583 us; error 29 us; skew 500 ppm
Aug 09 16:11:17 dist-test-slave-fgb3 krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:17.817051 20231 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:17.817512 20231 webserver.cc:460] Webserver started at http://127.16.224.129:35233/ using document root <none> and password file <none>
I20260809 16:11:17.817719 20231 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:17.817763 20231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:17.817894 20231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:17.819093 20231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "922c00cd71d448f89c6f1af4580b615e"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "6ff193878d31b179960805666faea019"
server_key_iv: "bd06d24616368e490bf23a269ef4d674"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.819486 20231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "922c00cd71d448f89c6f1af4580b615e"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "6ff193878d31b179960805666faea019"
server_key_iv: "bd06d24616368e490bf23a269ef4d674"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.820880 20231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:17.821642 20253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:17.821799 20231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:17.821887 20231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "922c00cd71d448f89c6f1af4580b615e"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "6ff193878d31b179960805666faea019"
server_key_iv: "bd06d24616368e490bf23a269ef4d674"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.821959 20231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:17.849509 20231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:17.851504 20231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:17.851640 20231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:17.852183 20231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:17.852229 20231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:17.852252 20231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:17.852264 20231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:17.859392 20231 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:33939
I20260809 16:11:17.859479 20459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:33939 every 8 connection(s)
Aug 09 16:11:17 dist-test-slave-fgb3 krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:17.859797 20231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:17.862303 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:17.852700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:44719 (local address 127.16.224.190:43391)
0809 16:11:17.852770 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:17.852772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:17.853019 (+ 247us) server_negotiation.cc:407] Connection header received
0809 16:11:17.853715 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:17.853716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:17.853727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:17.853765 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:17.854688 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.855195 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.856658 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.856978 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.857048 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:17.860185 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:17.860187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:17.860187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:17.860195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:17.861266 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:17.861618 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:17.861619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:17.861619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:17.861662 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:17.861835 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:17.861835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:17.861835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:17.861936 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:17.861980 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:17.862174 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:11:17.862239 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:11:17.862483 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:17.852666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43391 (local address 127.16.224.129:44719)
0809 16:11:17.852877 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:17.852929 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:17.853597 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:17.853878 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:17.853882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:17.854240 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:17.854584 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:17.854592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.855273 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.855274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:17.856549 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:17.856553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.857079 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.857080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:17.857235 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:17.858273 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:17.858284 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:17.860077 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:17.861331 (+ 1254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:17.861338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:17.861340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:17.861560 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:17.861706 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:17.861706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:17.861707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:17.861795 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:17.861977 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:17.861980 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:17.862088 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 16:11:17.862160 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:11:17.862274 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260809 16:11:17.862959 20460 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43391
I20260809 16:11:17.863039 20460 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:17.863205 20460 heartbeater.cc:511] Master 127.16.224.190:43391 requested a full tablet report, sending...
I20260809 16:11:17.863757 20102 ts_manager.cc:194] Registered new tserver with Master: 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939)
I20260809 16:11:17.864538 20102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:44719
I20260809 16:11:17.865821 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20231
I20260809 16:11:17.865896 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:17.866076 17282 external_mini_cluster.cc:1428] Setting key 45dbb9ada71b9b53bc222f4c45848a33
I20260809 16:11:17.869319 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:17.863944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:50511 (local address 127.16.224.190:43391)
0809 16:11:17.863985 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:17.863987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:17.864090 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:11:17.864115 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:17.864116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:17.864126 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:17.864185 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:17.864423 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.865043 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.865772 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.866021 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.866077 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:17.867021 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:17.867022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:17.867023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:17.867030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:17.868301 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:17.868647 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:17.868648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:17.868648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:17.868700 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:17.868862 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:17.868862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:17.868869 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:17.868965 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:17.869019 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:17.869162 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 16:11:17.869230 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:17.869333 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:17.863898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43391 (local address 127.16.224.129:50511)
0809 16:11:17.864032 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:17.864040 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:17.864086 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:17.864189 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:17.864190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:17.864322 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:17.864374 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:17.864375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.865132 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.865133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:17.865693 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:17.865695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:17.866086 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:17.866087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:17.866164 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:17.866567 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:17.866572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:17.866946 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:17.868383 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:17.868384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:17.868384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:17.868575 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:17.868754 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:17.868755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:17.868756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:17.868812 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:17.869019 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:17.869020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:17.869076 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 16:11:17.869133 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:11:17.869230 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:17.888723 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43391
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:17.969720 20478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:17.969944 20478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:17.969969 20478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:17.971117 20478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:17.971159 20478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:17.971179 20478 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:11:17.971189 20478 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:17.971204 20478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:17.972666 20478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:43391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:17.972860 20478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:17.973047 20478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:17.977797 20487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:17.977842 20489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:17.977805 20486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:17.978171 20478 server_base.cc:1034] running on GCE node
I20260809 16:11:17.978405 20478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:17.978622 20478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:17.979755 20478 hybrid_clock.cc:648] HybridClock initialized: now 1786291877979743 us; error 29 us; skew 500 ppm
Aug 09 16:11:17 dist-test-slave-fgb3 krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:17.982163 20478 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:17.982661 20478 webserver.cc:460] Webserver started at http://127.16.224.130:46777/ using document root <none> and password file <none>
I20260809 16:11:17.982906 20478 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:17.982952 20478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:17.983047 20478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:17.984076 20478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "5263387acd3f43d2a914a90fb331a96d"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "b66a2b871a910ec28332485efa84fb76"
server_key_iv: "2d903ac2c858760a972067b73e43bca0"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.984418 20478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "5263387acd3f43d2a914a90fb331a96d"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "b66a2b871a910ec28332485efa84fb76"
server_key_iv: "2d903ac2c858760a972067b73e43bca0"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.985682 20478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:17.986395 20499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:17.986554 20478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:17.986642 20478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "5263387acd3f43d2a914a90fb331a96d"
format_stamp: "Formatted at 2026-08-09 16:11:17 on dist-test-slave-fgb3"
server_key: "b66a2b871a910ec28332485efa84fb76"
server_key_iv: "2d903ac2c858760a972067b73e43bca0"
server_key_version: "encryptionkey@0"
I20260809 16:11:17.986706 20478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:17.998185 20478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:18.000211 20478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:18.000366 20478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:18.000914 20478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:18.000957 20478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.000985 20478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:18.001003 20478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:11:18 dist-test-slave-fgb3 krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:18.008010 20478 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:40341
I20260809 16:11:18.008096 20698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:40341 every 8 connection(s)
I20260809 16:11:18.008451 20478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:18.009873 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.001306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:32831 (local address 127.16.224.190:43391)
0809 16:11:18.001370 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.001373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.001545 (+ 172us) server_negotiation.cc:407] Connection header received
0809 16:11:18.002028 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.002029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.002044 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.002120 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:18.002771 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.003314 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.004315 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.004525 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.004580 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.007048 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:18.007050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:18.007051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:18.007059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:18.008578 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.008986 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.008988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.008988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.009055 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.009271 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.009272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.009272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.009387 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:18.009444 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.009701 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.009785 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:11:18.010030 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.001264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43391 (local address 127.16.224.130:32831)
0809 16:11:18.001418 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.001459 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.001935 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.002187 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.002191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.002481 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:18.002680 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.002686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.003400 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.003401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.004233 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.004236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.004595 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.004596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.004714 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.005176 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:18.005184 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:18.006965 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:18.008648 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.008655 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.008658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.008916 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.009109 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.009110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.009111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.009214 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.009443 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:18.009449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:18.009581 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.009669 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.009821 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260809 16:11:18.010504 20699 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43391
I20260809 16:11:18.010581 20699 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:18.010748 20699 heartbeater.cc:511] Master 127.16.224.190:43391 requested a full tablet report, sending...
I20260809 16:11:18.011111 20103 ts_manager.cc:194] Registered new tserver with Master: 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341)
I20260809 16:11:18.011514 20103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:32831
I20260809 16:11:18.013213 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20478
I20260809 16:11:18.013299 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:18.013463 17282 external_mini_cluster.cc:1428] Setting key 9c4001ad30bb24e8a9186274d0aed15c
I20260809 16:11:18.016597 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.011519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:51577 (local address 127.16.224.190:43391)
0809 16:11:18.011561 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.011563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.011633 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:11:18.011714 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.011715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.011723 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.011758 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:18.012066 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.012522 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.013182 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.013414 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.013468 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.014293 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:18.014294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:18.014295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:18.014302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:18.015578 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.015876 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.015877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.015877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.015938 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.016152 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.016152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.016153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.016296 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:18.016347 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.016482 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.016536 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:11:18.016662 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.011443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43391 (local address 127.16.224.130:51577)
0809 16:11:18.011566 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.011577 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.011665 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.011764 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.011765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.011935 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:18.011992 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.011994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.012609 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.012610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.013113 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.013114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.013477 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.013478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.013550 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.013886 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:18.013890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:18.014215 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:18.015663 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.015664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.015664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.015818 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.016019 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.016020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.016020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.016091 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.016351 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:18.016352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:18.016398 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.016465 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.016562 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:18.038118 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43391
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:18.118252 20712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:18.118510 20712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:18.118546 20712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:18.119669 20712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:18.119710 20712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:18.119737 20712 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:11:18.119753 20712 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:18.119776 20712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:18.121317 20712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:43391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:18.121568 20712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:18.121774 20712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:11:18.126776 20712 server_base.cc:1034] running on GCE node
W20260809 16:11:18.126760 20725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:18.126748 20722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:18.126758 20719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:18.127120 20712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:18.127338 20712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:18.128472 20712 hybrid_clock.cc:648] HybridClock initialized: now 1786291878128460 us; error 28 us; skew 500 ppm
Aug 09 16:11:18 dist-test-slave-fgb3 krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291878, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:18.130872 20712 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:18.131276 20712 webserver.cc:460] Webserver started at http://127.16.224.131:36611/ using document root <none> and password file <none>
I20260809 16:11:18.131505 20712 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:18.131548 20712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:18.131672 20712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:18.132683 20712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "54656118cb234510b010a4eef8567eea"
format_stamp: "Formatted at 2026-08-09 16:11:18 on dist-test-slave-fgb3"
server_key: "c6871236e453b24ee7b4b6fbe1953e32"
server_key_iv: "2e0647ced50af443aa903aba203c5a73"
server_key_version: "encryptionkey@0"
I20260809 16:11:18.133018 20712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "54656118cb234510b010a4eef8567eea"
format_stamp: "Formatted at 2026-08-09 16:11:18 on dist-test-slave-fgb3"
server_key: "c6871236e453b24ee7b4b6fbe1953e32"
server_key_iv: "2e0647ced50af443aa903aba203c5a73"
server_key_version: "encryptionkey@0"
I20260809 16:11:18.134197 20712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:11:18.134892 20741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.135030 20712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.135114 20712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "54656118cb234510b010a4eef8567eea"
format_stamp: "Formatted at 2026-08-09 16:11:18 on dist-test-slave-fgb3"
server_key: "c6871236e453b24ee7b4b6fbe1953e32"
server_key_iv: "2e0647ced50af443aa903aba203c5a73"
server_key_version: "encryptionkey@0"
I20260809 16:11:18.135180 20712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:18.146076 20712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:18.148054 20712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:18.148187 20712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:18.148664 20712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:18.148707 20712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.148738 20712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:18.148765 20712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:11:18 dist-test-slave-fgb3 krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291878, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:18.155627 20712 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:44659
I20260809 16:11:18.155966 20712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:18.155983 20935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:44659 every 8 connection(s)
I20260809 16:11:18.157920 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.149165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:47293 (local address 127.16.224.190:43391)
0809 16:11:18.149235 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.149237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.149452 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:11:18.149911 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.149912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.149928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.150008 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:18.150752 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.151507 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.152703 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.153008 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.153091 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.155177 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:18.155178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:18.155179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:18.155187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:18.156587 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.157012 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.157013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.157014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.157073 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.157288 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.157288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.157289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.157424 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:18.157487 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.157752 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.157832 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:11:18.158095 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.149103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43391 (local address 127.16.224.131:47293)
0809 16:11:18.149353 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.149381 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.149824 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.150070 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.150073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.150444 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:18.150662 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.150670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.151606 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.151607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.152608 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.152612 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.153109 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.153110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.153247 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.153755 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:18.153763 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:18.155084 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:18.156660 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.156667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.156669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.156945 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.157124 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.157125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.157125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.157231 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.157492 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:18.157499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:18.157630 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.157717 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.157879 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260809 16:11:18.158567 20936 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43391
I20260809 16:11:18.158651 20936 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:18.158819 20936 heartbeater.cc:511] Master 127.16.224.190:43391 requested a full tablet report, sending...
I20260809 16:11:18.159194 20102 ts_manager.cc:194] Registered new tserver with Master: 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:18.159585 20102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:47293
I20260809 16:11:18.162626 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20712
I20260809 16:11:18.162709 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:18.162884 17282 external_mini_cluster.cc:1428] Setting key ecad381cce799864cd9e9cd1cbbf1418
I20260809 16:11:18.163718 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:11:18.164680 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.159540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:35301 (local address 127.16.224.190:43391)
0809 16:11:18.159580 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.159582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.159664 (+ 82us) server_negotiation.cc:407] Connection header received
0809 16:11:18.159716 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.159716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.159727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.159777 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:18.160067 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.160537 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.161198 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.161400 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.161451 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.162228 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:18.162230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:18.162230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:18.162237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:18.163675 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.163985 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.163986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.163986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.164067 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.164226 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.164227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.164228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.164330 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:18.164386 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.164527 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.164591 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:18.164685 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.159472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43391 (local address 127.16.224.131:35301)
0809 16:11:18.159597 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.159607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.159660 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.159781 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.159782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.159957 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:18.160004 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.160006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.160622 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.160623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.161129 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.161131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.161457 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.161458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.161538 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.161895 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:18.161900 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:18.162150 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:18.163739 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.163740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.163740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.163921 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.164121 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.164121 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.164122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.164177 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.164384 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:18.164385 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:18.164431 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.164497 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.164594 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
Aug 09 16:11:18 dist-test-slave-fgb3 krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:11:18 dist-test-slave-fgb3 krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:18.186952 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.180466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.16.224.190:43391)
0809 16:11:18.180543 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.180545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.180555 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:18.180581 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.180582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.180592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.180645 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:18.181181 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.181690 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.182443 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.182633 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.182686 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.184853 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:18.184855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:18.184855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:18.184863 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:18.186044 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.186374 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.186375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.186376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.186431 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.186577 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.186578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.186578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.186679 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:18.186749 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.186825 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.186884 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:11:18.188545 20102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:18.190161 20102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:18.196458 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.192435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44659 (local address 127.0.0.1:60254)
0809 16:11:18.192738 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.192749 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.192801 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.193480 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.193481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.193483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:18.193517 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.193518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.194690 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.194691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.195527 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.195529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.196177 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.196179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.196256 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.196270 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.196330 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.196382 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":85,"threads_started":1}
I20260809 16:11:18.196489 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.192619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.16.224.131:44659)
0809 16:11:18.193281 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.193286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.193294 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:18.193328 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.193331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.193346 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.193414 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:18.193596 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.194623 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.195598 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.196104 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.196169 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.196229 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.196359 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.196396 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":85,"threads_started":1}
I20260809 16:11:18.196689 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.192426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40341 (local address 127.0.0.1:48068)
0809 16:11:18.192640 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.192687 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.192749 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.193174 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.193177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.193180 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:18.193257 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.193259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.194511 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.194512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.195553 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.195555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.196484 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.196485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.196546 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.196549 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.196580 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.196616 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":57,"threads_started":1}
I20260809 16:11:18.196817 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.192627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:33939 (local address 127.0.0.1:40400)
0809 16:11:18.192839 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.192847 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.192882 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.193236 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.193237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.193239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:18.193295 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.193297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.194219 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.194220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.195541 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.195544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.196625 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.196626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.196682 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.196686 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.196727 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.196763 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":69,"threads_started":1}
I20260809 16:11:18.197083 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.192567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.16.224.130:40341)
0809 16:11:18.192996 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.193001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.193011 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:18.193050 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.193053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.193069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.193154 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:18.193321 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.194419 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.195652 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.196378 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.196841 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.196917 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.196944 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.196986 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":79,"threads_started":1}
I20260809 16:11:18.197098 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.192860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40400 (local address 127.16.224.129:33939)
0809 16:11:18.193017 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.193021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.193029 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:18.193065 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.193067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.193103 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.193166 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:18.193371 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.194138 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.195652 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.196368 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.196440 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.196501 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.196973 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.197013 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260809 16:11:18.197322 20826 tablet_service.cc:1467] Processing CreateTablet for tablet a4ad82946dde46e19add9cdaa5d12ca5 (DEFAULT_TABLE table=test-table [id=fc7afea6dbc84c468b7a07dd10672e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:18.197736 20826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ad82946dde46e19add9cdaa5d12ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.197690 20580 tablet_service.cc:1467] Processing CreateTablet for tablet a4ad82946dde46e19add9cdaa5d12ca5 (DEFAULT_TABLE table=test-table [id=fc7afea6dbc84c468b7a07dd10672e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:18.197711 20345 tablet_service.cc:1467] Processing CreateTablet for tablet a4ad82946dde46e19add9cdaa5d12ca5 (DEFAULT_TABLE table=test-table [id=fc7afea6dbc84c468b7a07dd10672e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:18.198055 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ad82946dde46e19add9cdaa5d12ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.198055 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ad82946dde46e19add9cdaa5d12ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.200489 20975 tablet_bootstrap.cc:492] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Bootstrap starting.
I20260809 16:11:18.201387 20975 tablet_bootstrap.cc:654] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.201707 20975 log.cc:824] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:18.201730 20977 tablet_bootstrap.cc:492] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Bootstrap starting.
I20260809 16:11:18.202389 20977 tablet_bootstrap.cc:654] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.202474 20976 tablet_bootstrap.cc:492] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Bootstrap starting.
I20260809 16:11:18.202507 20975 tablet_bootstrap.cc:492] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: No bootstrap required, opened a new log
I20260809 16:11:18.202564 20975 ts_tablet_manager.cc:1397] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:18.202713 20977 log.cc:824] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:18.203080 20976 tablet_bootstrap.cc:654] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.203372 20977 tablet_bootstrap.cc:492] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: No bootstrap required, opened a new log
I20260809 16:11:18.203449 20977 ts_tablet_manager.cc:1397] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:11:18.203603 20976 log.cc:824] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:18.204427 20976 tablet_bootstrap.cc:492] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: No bootstrap required, opened a new log
I20260809 16:11:18.204499 20976 ts_tablet_manager.cc:1397] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:18.204435 20975 raft_consensus.cc:348] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.204604 20975 raft_consensus.cc:374] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.204645 20975 raft_consensus.cc:729] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54656118cb234510b010a4eef8567eea, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.204766 20975 consensus_queue.cc:260] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.205042 20975 ts_tablet_manager.cc:1428] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:11:18.205173 20936 heartbeater.cc:503] Master 127.16.224.190:43391 was elected leader, sending a full tablet report...
I20260809 16:11:18.205478 20977 raft_consensus.cc:348] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.205641 20977 raft_consensus.cc:374] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.205685 20977 raft_consensus.cc:729] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922c00cd71d448f89c6f1af4580b615e, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.205787 20977 consensus_queue.cc:260] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.206086 20460 heartbeater.cc:503] Master 127.16.224.190:43391 was elected leader, sending a full tablet report...
I20260809 16:11:18.206130 20977 ts_tablet_manager.cc:1428] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:11:18.206522 20976 raft_consensus.cc:348] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.206640 20976 raft_consensus.cc:374] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.206670 20976 raft_consensus.cc:729] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.206780 20976 consensus_queue.cc:260] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.207039 20976 ts_tablet_manager.cc:1428] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:11:18.207167 20699 heartbeater.cc:503] Master 127.16.224.190:43391 was elected leader, sending a full tablet report...
W20260809 16:11:18.259024 20701 tablet.cc:2369] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:18.262183 20984 raft_consensus.cc:482] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:18.262297 20984 raft_consensus.cc:504] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.262701 20984 leader_election.cc:288] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54656118cb234510b010a4eef8567eea (127.16.224.131:44659), 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341)
I20260809 16:11:18.265540 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.262943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:50685 (local address 127.16.224.131:44659)
0809 16:11:18.262989 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.262992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.263003 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:18.263068 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.263069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.263077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.263125 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:18.263324 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.263924 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.264850 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.265220 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.265274 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.265292 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.265444 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.265477 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:18.265542 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.262962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:42795 (local address 127.16.224.130:40341)
0809 16:11:18.263020 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.263022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.263303 (+ 281us) server_negotiation.cc:407] Connection header received
0809 16:11:18.263334 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.263335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.263344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.263392 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:18.263527 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.263978 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.264871 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.265233 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.265298 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.265306 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.265439 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.265475 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:11:18.265558 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.262889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44659 (local address 127.16.224.129:50685)
0809 16:11:18.262954 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.262964 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.263008 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.263158 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.263159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.263162 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:18.263241 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.263243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.264002 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.264003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.264767 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.264770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.265298 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.265300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.265380 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.265383 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.265432 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.265475 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":3}
I20260809 16:11:18.265573 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.262892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40341 (local address 127.16.224.129:42795)
0809 16:11:18.263229 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.263237 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.263288 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.263407 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.263408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.263410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:18.263472 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.263474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.264046 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.264046 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.264770 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.264772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.265313 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.265314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.265373 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.265376 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.265419 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.265465 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":98,"thread_start_us":136,"threads_started":1}
I20260809 16:11:18.265893 20616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ad82946dde46e19add9cdaa5d12ca5" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5263387acd3f43d2a914a90fb331a96d" is_pre_election: true
I20260809 16:11:18.266024 20616 raft_consensus.cc:2454] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 0.
I20260809 16:11:18.266206 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ad82946dde46e19add9cdaa5d12ca5" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea" is_pre_election: true
I20260809 16:11:18.266228 20261 leader_election.cc:302] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:18.266351 20858 raft_consensus.cc:2454] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 0.
I20260809 16:11:18.266433 20984 raft_consensus.cc:2781] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:18.266501 20984 raft_consensus.cc:482] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:18.266530 20984 raft_consensus.cc:3037] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.267232 20984 raft_consensus.cc:504] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.267418 20984 leader_election.cc:288] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 election: Requested vote from peers 54656118cb234510b010a4eef8567eea (127.16.224.131:44659), 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341)
I20260809 16:11:18.267500 20616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ad82946dde46e19add9cdaa5d12ca5" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5263387acd3f43d2a914a90fb331a96d"
I20260809 16:11:18.267587 20616 raft_consensus.cc:3037] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.268339 20616 raft_consensus.cc:2454] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 1.
I20260809 16:11:18.268446 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ad82946dde46e19add9cdaa5d12ca5" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea"
I20260809 16:11:18.268476 20261 leader_election.cc:302] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:18.268523 20858 raft_consensus.cc:3037] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.268560 20984 raft_consensus.cc:2781] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:18.268710 20984 raft_consensus.cc:686] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 1 LEADER]: Becoming Leader. State: Replica: 922c00cd71d448f89c6f1af4580b615e, State: Running, Role: LEADER
I20260809 16:11:18.268800 20984 consensus_queue.cc:237] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } }
I20260809 16:11:18.269341 20858 raft_consensus.cc:2454] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 1.
I20260809 16:11:18.269692 20103 catalog_manager.cc:5495] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e reported cstate change: term changed from 0 to 1, leader changed from <none> to 922c00cd71d448f89c6f1af4580b615e (127.16.224.129). New cstate: current_term: 1 leader_uuid: "922c00cd71d448f89c6f1af4580b615e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } health_report { overall_health: UNKNOWN } } }
Aug 09 16:11:18 dist-test-slave-fgb3 krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:18.313496 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.307007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43391 (local address 127.0.0.1:37184)
0809 16:11:18.307201 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.307212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.307278 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.307408 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.307409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.307733 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:18.307782 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.307795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.308418 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.308419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.308951 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.308952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.309254 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.309255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.309304 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.309799 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:18.309803 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:18.311262 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:18.312617 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.312625 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.312628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.312830 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.312983 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:18.312983 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:18.312983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:18.313033 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:18.313208 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:18.313213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:18.313255 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.313322 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.313420 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260809 16:11:18.313567 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.307091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.16.224.190:43391)
0809 16:11:18.307154 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.307156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.307269 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:11:18.307337 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.307338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.307349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.307398 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:18.307871 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.308350 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.309024 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.309197 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.309250 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.311381 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:18.311382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:18.311382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:18.311389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:18.312544 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.312894 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.312895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.312895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.312942 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:18.313073 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:18.313073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:18.313073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:18.313169 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:18.313212 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.313354 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.313469 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":30}
I20260809 16:11:18.315467 20103 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:37184:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:11:18.315675 20103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:18.317198 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 2c2629d98f60411f8e6dab64a003df28 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=adabba3b675147b1b2dfa44f8e6f092c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:18.317198 20826 tablet_service.cc:1467] Processing CreateTablet for tablet 2c2629d98f60411f8e6dab64a003df28 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=adabba3b675147b1b2dfa44f8e6f092c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:18.317304 20345 tablet_service.cc:1467] Processing CreateTablet for tablet 2c2629d98f60411f8e6dab64a003df28 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=adabba3b675147b1b2dfa44f8e6f092c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:18.317392 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c2629d98f60411f8e6dab64a003df28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.317392 20826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c2629d98f60411f8e6dab64a003df28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.317499 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c2629d98f60411f8e6dab64a003df28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.318832 20977 tablet_bootstrap.cc:492] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: Bootstrap starting.
I20260809 16:11:18.318924 20976 tablet_bootstrap.cc:492] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: Bootstrap starting.
I20260809 16:11:18.318943 20975 tablet_bootstrap.cc:492] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: Bootstrap starting.
I20260809 16:11:18.319330 20977 tablet_bootstrap.cc:654] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.319399 20975 tablet_bootstrap.cc:654] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.319558 20976 tablet_bootstrap.cc:654] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.320026 20975 tablet_bootstrap.cc:492] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: No bootstrap required, opened a new log
I20260809 16:11:18.320086 20975 ts_tablet_manager.cc:1397] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:18.320089 20977 tablet_bootstrap.cc:492] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: No bootstrap required, opened a new log
I20260809 16:11:18.320134 20977 ts_tablet_manager.cc:1397] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:18.320240 20975 raft_consensus.cc:348] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.320323 20975 raft_consensus.cc:374] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.320293 20977 raft_consensus.cc:348] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.320343 20975 raft_consensus.cc:729] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54656118cb234510b010a4eef8567eea, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.320358 20977 raft_consensus.cc:374] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.320382 20977 raft_consensus.cc:729] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922c00cd71d448f89c6f1af4580b615e, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.320387 20975 consensus_queue.cc:260] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.320417 20977 consensus_queue.cc:260] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.320478 20975 ts_tablet_manager.cc:1428] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.320513 20977 ts_tablet_manager.cc:1428] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.320598 20982 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.320703 20982 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:18.320729 20984 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.320813 20984 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:18.320894 20976 tablet_bootstrap.cc:492] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: No bootstrap required, opened a new log
I20260809 16:11:18.320928 20976 ts_tablet_manager.cc:1397] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:11:18.321038 20976 raft_consensus.cc:348] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.321080 20976 raft_consensus.cc:374] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.321092 20976 raft_consensus.cc:729] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.321132 20976 consensus_queue.cc:260] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.321233 20976 ts_tablet_manager.cc:1428] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.321393 20986 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.321468 20986 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 16:11:18.360628 20462 tablet.cc:2369] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:11:18.406535 20939 tablet.cc:2369] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:18.492982 20986 raft_consensus.cc:482] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:18.493093 20986 raft_consensus.cc:504] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.493499 20986 leader_election.cc:288] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:18.497395 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.493890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:35819 (local address 127.16.224.131:44659)
0809 16:11:18.493958 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.493960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.493969 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:18.494017 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.494018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.494026 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.494070 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:18.494207 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.494934 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.496464 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.497012 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.497083 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.497092 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.497297 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.497336 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:11:18.497388 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.493773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:51455 (local address 127.16.224.129:33939)
0809 16:11:18.493852 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.493854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.493865 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:18.493901 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.493902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.493912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.493973 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:18.494146 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.494934 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.496467 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.497010 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.497083 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.497092 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.497274 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.497312 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 16:11:18.497407 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.493692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:33939 (local address 127.16.224.130:51455)
0809 16:11:18.493746 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.493758 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.493825 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.494003 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.494004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.494006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:18.494079 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.494081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.495030 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.495032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.496369 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.496373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.497090 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.497092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.497190 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.497198 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.497264 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.497311 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":62}
I20260809 16:11:18.497421 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.493801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44659 (local address 127.16.224.130:35819)
0809 16:11:18.493922 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.493930 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.493966 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.494081 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.494082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.494083 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:18.494143 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.494145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.495030 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.495032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.496369 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.496373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.497113 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.497114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.497190 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.497198 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.497291 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.497323 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":51,"thread_start_us":62,"threads_started":1}
I20260809 16:11:18.497790 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c2629d98f60411f8e6dab64a003df28" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c00cd71d448f89c6f1af4580b615e" is_pre_election: true
I20260809 16:11:18.497908 20391 raft_consensus.cc:2454] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 0.
I20260809 16:11:18.498112 20504 leader_election.cc:302] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:18.498172 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c2629d98f60411f8e6dab64a003df28" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea" is_pre_election: true
I20260809 16:11:18.498247 20986 raft_consensus.cc:2781] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:18.498270 20858 raft_consensus.cc:2454] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 0.
I20260809 16:11:18.498287 20986 raft_consensus.cc:482] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:18.498337 20986 raft_consensus.cc:3037] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.499213 20986 raft_consensus.cc:504] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.499348 20986 leader_election.cc:288] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 election: Requested vote from peers 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:18.499548 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c2629d98f60411f8e6dab64a003df28" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c00cd71d448f89c6f1af4580b615e"
I20260809 16:11:18.499568 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c2629d98f60411f8e6dab64a003df28" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea"
I20260809 16:11:18.499622 20391 raft_consensus.cc:3037] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.499639 20858 raft_consensus.cc:3037] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.500257 20858 raft_consensus.cc:2454] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 1.
I20260809 16:11:18.500283 20391 raft_consensus.cc:2454] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 1.
I20260809 16:11:18.500428 20504 leader_election.cc:302] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 54656118cb234510b010a4eef8567eea; no voters:
I20260809 16:11:18.500540 20986 raft_consensus.cc:2781] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:18.500711 20986 raft_consensus.cc:686] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [term 1 LEADER]: Becoming Leader. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Running, Role: LEADER
I20260809 16:11:18.500823 20986 consensus_queue.cc:237] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.501255 21014 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: TxnStatusTablet state changed. Reason: New leader 5263387acd3f43d2a914a90fb331a96d. Latest consensus state: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.501341 21014 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:18.501549 21016 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:11:18.501679 21016 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:11:18.501698 20103 catalog_manager.cc:5495] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5263387acd3f43d2a914a90fb331a96d (127.16.224.130). New cstate: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:18.548513 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.546072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.16.224.130:40341)
0809 16:11:18.546138 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.546140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.546149 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:18.546190 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.546191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.546211 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.546270 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:18.546416 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.546893 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.547626 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.547825 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.547880 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.548138 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.548253 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.548326 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.548377 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.548398 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:11:18.548493 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.545993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40341 (local address 127.0.0.1:48070)
0809 16:11:18.546058 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.546069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.546116 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.546279 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.546280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.546281 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:18.546356 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.546358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.546979 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.546980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.547548 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.547550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.547885 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.547886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.548021 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.548044 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.548313 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.548321 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.548371 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.548422 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:11:18.550470 20858 raft_consensus.cc:1261] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Refusing update from remote peer 5263387acd3f43d2a914a90fb331a96d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:18.550470 20391 raft_consensus.cc:1261] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Refusing update from remote peer 5263387acd3f43d2a914a90fb331a96d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:18.550813 21014 consensus_queue.cc:1035] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Connected to new peer: Peer: permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:18.550951 20986 consensus_queue.cc:1035] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Connected to new peer: Peer: permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:18.552122 20984 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: TxnStatusTablet state changed. Reason: New leader 5263387acd3f43d2a914a90fb331a96d. Latest consensus state: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.552224 20984 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:18.552198 20982 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: TxnStatusTablet state changed. Reason: New leader 5263387acd3f43d2a914a90fb331a96d. Latest consensus state: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.552321 20982 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:18.553869 20986 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.553962 20986 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:18.554014 20986 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.554061 20986 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 5263387acd3f43d2a914a90fb331a96d: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:18.554118 20984 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.554196 20984 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 922c00cd71d448f89c6f1af4580b615e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:18.554623 20982 tablet_replica.cc:439] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } } }
I20260809 16:11:18.554710 20982 tablet_replica.cc:442] T 2c2629d98f60411f8e6dab64a003df28 P 54656118cb234510b010a4eef8567eea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:18.554965 21020 mvcc.cc:205] Tried to move back new op lower bound from 7316651534539960320 to 7316651534340022272. Current Snapshot: MvccSnapshot[applied={T|T < 7316651534539960320 or (T in {7316651534539960320})}]
I20260809 16:11:18.555073 21023 mvcc.cc:205] Tried to move back new op lower bound from 7316651534539960320 to 7316651534340022272. Current Snapshot: MvccSnapshot[applied={T|T < 7316651534539960320}]
I20260809 16:11:18.558638 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.556324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.16.224.129:33939)
0809 16:11:18.556372 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.556374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.556381 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:18.556412 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.556413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.556428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.556490 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:18.556619 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.557087 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.557826 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.558094 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.558145 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.558327 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.558444 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.558514 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.558550 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.558572 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:11:18.563455 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.561072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40341 (local address 127.16.224.129:36853)
0809 16:11:18.561248 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.561257 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.561312 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.561422 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.561423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.561424 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:18.561482 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.561483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.562047 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.562048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.562658 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.562660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.563040 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.563041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.563108 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.563124 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.563291 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.563293 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.563329 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.563378 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260809 16:11:18.563432 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.561135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:36853 (local address 127.16.224.130:40341)
0809 16:11:18.561174 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.561175 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.561316 (+ 141us) server_negotiation.cc:407] Connection header received
0809 16:11:18.561357 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.561358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.561368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.561418 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:18.561528 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.561980 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.562738 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.562981 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.563033 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.563207 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.563241 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.563285 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.563342 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.563361 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:11:18.565568 20858 raft_consensus.cc:1261] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Refusing update from remote peer 922c00cd71d448f89c6f1af4580b615e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:18.565614 20616 raft_consensus.cc:1261] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Refusing update from remote peer 922c00cd71d448f89c6f1af4580b615e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:18.565832 20984 consensus_queue.cc:1035] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [LEADER]: Connected to new peer: Peer: permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:18.565981 20990 consensus_queue.cc:1035] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:18.567889 21025 mvcc.cc:205] Tried to move back new op lower bound from 7316651534602838016 to 7316651533389680640. Current Snapshot: MvccSnapshot[applied={T|T < 7316651534602838016}]
I20260809 16:11:18.574796 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.572038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:59057 (local address 127.16.224.129:33939)
0809 16:11:18.572077 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.572079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.572207 (+ 128us) server_negotiation.cc:407] Connection header received
0809 16:11:18.572262 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.572263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.572274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.572325 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:18.572484 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.572943 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.574044 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.574239 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.574288 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.574561 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.574599 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.574640 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.574715 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.574738 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:11:18.575068 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.571985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:33939 (local address 127.16.224.130:59057)
0809 16:11:18.572142 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.572154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.572214 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.572338 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.572339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.572341 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:18.572415 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.572418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.573027 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.573028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.573967 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.573971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.574356 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.574357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.574423 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.574442 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.574663 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.574665 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.574712 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.574758 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260809 16:11:18.584803 20103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:18.584936 20103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:18.585729 20103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Sending DeleteTablet for 3 replicas of tablet a4ad82946dde46e19add9cdaa5d12ca5
I20260809 16:11:18.586113 20345 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ad82946dde46e19add9cdaa5d12ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:18 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:40400
I20260809 16:11:18.586120 20826 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ad82946dde46e19add9cdaa5d12ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:18 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:60254
I20260809 16:11:18.586516 21051 tablet_replica.cc:330] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: stopping tablet replica
I20260809 16:11:18.586505 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ad82946dde46e19add9cdaa5d12ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:18 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260809 16:11:18.586688 21051 raft_consensus.cc:2229] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:18.586732 20103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:18.586901 20103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:18.586833 21051 raft_consensus.cc:2258] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:18.587204 21052 tablet_replica.cc:330] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: stopping tablet replica
I20260809 16:11:18.587323 21052 raft_consensus.cc:2229] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:18.587378 21052 raft_consensus.cc:2258] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:18.588358 20345 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 (DEFAULT_TABLE table=test-table [id=bc196c5e74fe46b5a9b4762ad10c04bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:18.588498 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.588686 21051 ts_tablet_manager.cc:1905] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:18.590013 20826 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 (DEFAULT_TABLE table=test-table [id=bc196c5e74fe46b5a9b4762ad10c04bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:18.590134 20826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.590734 21051 ts_tablet_manager.cc:1918] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:18.590857 21051 log.cc:1196] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/a4ad82946dde46e19add9cdaa5d12ca5
I20260809 16:11:18.591202 21051 ts_tablet_manager.cc:1939] T a4ad82946dde46e19add9cdaa5d12ca5 P 922c00cd71d448f89c6f1af4580b615e: Deleting consensus metadata
I20260809 16:11:18.591512 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 (DEFAULT_TABLE table=test-table [id=bc196c5e74fe46b5a9b4762ad10c04bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:18.591634 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:18.592934 20977 tablet_bootstrap.cc:492] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: Bootstrap starting.
I20260809 16:11:18.593343 20977 tablet_bootstrap.cc:654] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.593667 20975 tablet_bootstrap.cc:492] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: Bootstrap starting.
I20260809 16:11:18.594002 20977 tablet_bootstrap.cc:492] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: No bootstrap required, opened a new log
I20260809 16:11:18.594049 20977 ts_tablet_manager.cc:1397] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:18.594067 20975 tablet_bootstrap.cc:654] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.594177 20977 raft_consensus.cc:348] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.594238 20977 raft_consensus.cc:374] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.594257 20977 raft_consensus.cc:729] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922c00cd71d448f89c6f1af4580b615e, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.594408 20976 tablet_bootstrap.cc:492] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: Bootstrap starting.
I20260809 16:11:18.594599 21053 tablet_replica.cc:330] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: stopping tablet replica
I20260809 16:11:18.594303 20977 consensus_queue.cc:260] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.594712 21053 raft_consensus.cc:2229] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:18.594740 20977 ts_tablet_manager.cc:1428] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:11:18.594762 21053 raft_consensus.cc:2258] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:18.594800 20976 tablet_bootstrap.cc:654] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:18.595482 20081 catalog_manager.cc:4841] TS 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939): tablet a4ad82946dde46e19add9cdaa5d12ca5 (table test-table [id=fc7afea6dbc84c468b7a07dd10672e1b]) successfully deleted
I20260809 16:11:18.596050 21053 ts_tablet_manager.cc:1905] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:18.596375 21052 ts_tablet_manager.cc:1905] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:18.597555 21053 ts_tablet_manager.cc:1918] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:18.597618 21053 log.cc:1196] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/a4ad82946dde46e19add9cdaa5d12ca5
I20260809 16:11:18.597872 21053 ts_tablet_manager.cc:1939] T a4ad82946dde46e19add9cdaa5d12ca5 P 5263387acd3f43d2a914a90fb331a96d: Deleting consensus metadata
I20260809 16:11:18.598327 20082 catalog_manager.cc:4841] TS 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341): tablet a4ad82946dde46e19add9cdaa5d12ca5 (table test-table [id=fc7afea6dbc84c468b7a07dd10672e1b]) successfully deleted
I20260809 16:11:18.598454 21052 ts_tablet_manager.cc:1918] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:18.598510 21052 log.cc:1196] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/a4ad82946dde46e19add9cdaa5d12ca5
I20260809 16:11:18.598737 21052 ts_tablet_manager.cc:1939] T a4ad82946dde46e19add9cdaa5d12ca5 P 54656118cb234510b010a4eef8567eea: Deleting consensus metadata
I20260809 16:11:18.599121 20081 catalog_manager.cc:4841] TS 54656118cb234510b010a4eef8567eea (127.16.224.131:44659): tablet a4ad82946dde46e19add9cdaa5d12ca5 (table test-table [id=fc7afea6dbc84c468b7a07dd10672e1b]) successfully deleted
I20260809 16:11:18.599671 20976 tablet_bootstrap.cc:492] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: No bootstrap required, opened a new log
I20260809 16:11:18.599726 20976 ts_tablet_manager.cc:1397] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260809 16:11:18.599885 20976 raft_consensus.cc:348] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.599977 20976 raft_consensus.cc:374] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.600003 20976 raft_consensus.cc:729] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.600049 20976 consensus_queue.cc:260] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.600135 20976 ts_tablet_manager.cc:1428] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:18.600687 20975 tablet_bootstrap.cc:492] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: No bootstrap required, opened a new log
I20260809 16:11:18.600737 20975 ts_tablet_manager.cc:1397] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260809 16:11:18.600883 20975 raft_consensus.cc:348] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.600960 20975 raft_consensus.cc:374] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:18.600980 20975 raft_consensus.cc:729] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54656118cb234510b010a4eef8567eea, State: Initialized, Role: FOLLOWER
I20260809 16:11:18.601024 20975 consensus_queue.cc:260] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.601114 20975 ts_tablet_manager.cc:1428] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:18.758530 21028 raft_consensus.cc:482] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:18.758657 21028 raft_consensus.cc:504] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.758823 21028 leader_election.cc:288] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:18.759111 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca3eb164a9f4cb8bfdc9b37b6b2c105" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea" is_pre_election: true
I20260809 16:11:18.759112 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca3eb164a9f4cb8bfdc9b37b6b2c105" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c00cd71d448f89c6f1af4580b615e" is_pre_election: true
I20260809 16:11:18.759231 20858 raft_consensus.cc:2454] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 0.
I20260809 16:11:18.759231 20391 raft_consensus.cc:2454] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 0.
I20260809 16:11:18.759474 20504 leader_election.cc:302] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:18.759606 21028 raft_consensus.cc:2781] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:18.759657 21028 raft_consensus.cc:482] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:18.759676 21028 raft_consensus.cc:3037] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.760471 21028 raft_consensus.cc:504] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.760609 21028 leader_election.cc:288] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 election: Requested vote from peers 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:18.760767 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca3eb164a9f4cb8bfdc9b37b6b2c105" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c00cd71d448f89c6f1af4580b615e"
I20260809 16:11:18.760849 20391 raft_consensus.cc:3037] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.760787 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca3eb164a9f4cb8bfdc9b37b6b2c105" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea"
I20260809 16:11:18.760883 20858 raft_consensus.cc:3037] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:18.761479 20391 raft_consensus.cc:2454] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 1.
I20260809 16:11:18.761520 20858 raft_consensus.cc:2454] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 1.
I20260809 16:11:18.761667 20504 leader_election.cc:302] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:18.761772 21028 raft_consensus.cc:2781] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:18.761847 21028 raft_consensus.cc:686] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 1 LEADER]: Becoming Leader. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Running, Role: LEADER
I20260809 16:11:18.761904 21028 consensus_queue.cc:237] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:18.762568 20103 catalog_manager.cc:5495] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5263387acd3f43d2a914a90fb331a96d (127.16.224.130). New cstate: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:18.780570 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.778571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.16.224.130:40341)
0809 16:11:18.778628 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.778630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.778638 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:18.778703 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.778704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.778715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.778756 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:18.778860 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.779368 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.780068 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.780263 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.780310 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.780413 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.780438 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.780465 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.780500 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.780516 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:11:18.784221 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.782392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:38275 (local address 127.16.224.130:40341)
0809 16:11:18.782434 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:18.782442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:18.782449 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:18.782468 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:18.782469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:18.782477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:18.782522 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:18.782594 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.783001 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.783728 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.783895 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.783952 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.784030 (+ 78us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.784051 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.784094 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:18.784145 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:11:18.784158 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:11:18.784325 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:18.782353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40341 (local address 127.16.224.130:38275)
0809 16:11:18.782386 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:18.782396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:18.782432 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:18.782523 (+ 91us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:18.782524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:18.782524 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:18.782555 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.782556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.783077 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.783079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.783667 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:18.783669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:18.783949 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:18.783950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:18.783982 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:18.783987 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:18.784096 (+ 109us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:18.784097 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:11:18.784141 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:18.784185 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260809 16:11:18.785933 20858 raft_consensus.cc:1261] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Refusing update from remote peer 5263387acd3f43d2a914a90fb331a96d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:18.785959 20391 raft_consensus.cc:1261] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Refusing update from remote peer 5263387acd3f43d2a914a90fb331a96d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:18.786144 21028 consensus_queue.cc:1035] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Connected to new peer: Peer: permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:18.786252 21014 consensus_queue.cc:1035] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Connected to new peer: Peer: permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:11:18.790817 21020 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:11:20.308676 20103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:20.308784 20103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:20.309666 20103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Sending DeleteTablet for 3 replicas of tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105
I20260809 16:11:20.309950 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:20 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260809 16:11:20.309962 20345 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:20 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:40400
I20260809 16:11:20.309989 20826 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:20 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:60254
I20260809 16:11:20.310506 21110 tablet_replica.cc:330] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: stopping tablet replica
I20260809 16:11:20.310572 21110 raft_consensus.cc:2229] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:20.310667 21108 tablet_replica.cc:330] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: stopping tablet replica
I20260809 16:11:20.310714 21108 raft_consensus.cc:2229] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:20.310810 21108 raft_consensus.cc:2258] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:20.310935 21110 raft_consensus.cc:2258] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:20.312456 21108 ts_tablet_manager.cc:1905] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:20.314221 21108 ts_tablet_manager.cc:1918] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:11:20.314286 21108 log.cc:1196] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/6ca3eb164a9f4cb8bfdc9b37b6b2c105
I20260809 16:11:20.314545 21108 ts_tablet_manager.cc:1939] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 5263387acd3f43d2a914a90fb331a96d: Deleting consensus metadata
I20260809 16:11:20.314837 21113 tablet_replica.cc:330] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: stopping tablet replica
I20260809 16:11:20.314865 20082 catalog_manager.cc:4841] TS 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341): tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 (table test-table [id=bc196c5e74fe46b5a9b4762ad10c04bb]) successfully deleted
I20260809 16:11:20.314904 21113 raft_consensus.cc:2229] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:20.314939 21113 raft_consensus.cc:2258] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:20.315117 21110 ts_tablet_manager.cc:1905] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:20.316704 21110 ts_tablet_manager.cc:1918] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:11:20.316768 21110 log.cc:1196] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/6ca3eb164a9f4cb8bfdc9b37b6b2c105
I20260809 16:11:20.316998 21110 ts_tablet_manager.cc:1939] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 54656118cb234510b010a4eef8567eea: Deleting consensus metadata
I20260809 16:11:20.317301 20081 catalog_manager.cc:4841] TS 54656118cb234510b010a4eef8567eea (127.16.224.131:44659): tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 (table test-table [id=bc196c5e74fe46b5a9b4762ad10c04bb]) successfully deleted
I20260809 16:11:20.317524 21113 ts_tablet_manager.cc:1905] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:20.318858 21113 ts_tablet_manager.cc:1918] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:11:20.318925 21113 log.cc:1196] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/6ca3eb164a9f4cb8bfdc9b37b6b2c105
I20260809 16:11:20.319116 21113 ts_tablet_manager.cc:1939] T 6ca3eb164a9f4cb8bfdc9b37b6b2c105 P 922c00cd71d448f89c6f1af4580b615e: Deleting consensus metadata
I20260809 16:11:20.319418 20081 catalog_manager.cc:4841] TS 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939): tablet 6ca3eb164a9f4cb8bfdc9b37b6b2c105 (table test-table [id=bc196c5e74fe46b5a9b4762ad10c04bb]) successfully deleted
Aug 09 16:11:20 dist-test-slave-fgb3 krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291880, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:11:20 dist-test-slave-fgb3 krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291880, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:20.338791 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:20.328991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.16.224.190:43391)
0809 16:11:20.329152 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:20.329154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:20.329163 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:20.329190 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:20.329191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:20.329204 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:20.329266 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:20.329821 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:20.330395 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:20.331122 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:20.331344 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:20.331399 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:20.336548 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:20.336550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:20.336550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:20.336557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:20.337791 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:20.338128 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:20.338129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:20.338129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:20.338183 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:20.338335 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:20.338336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:20.338336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:20.338455 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:20.338529 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:20.338642 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 16:11:20.338718 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260809 16:11:20.340126 20103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:20.340298 20103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:20.341638 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 40b2046b97eb40dea7a35a406566cd0b (DEFAULT_TABLE table=test-table [id=5bc94a862237402186e8177fcc404de0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:20.341634 20826 tablet_service.cc:1467] Processing CreateTablet for tablet 40b2046b97eb40dea7a35a406566cd0b (DEFAULT_TABLE table=test-table [id=5bc94a862237402186e8177fcc404de0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:20.341631 20345 tablet_service.cc:1467] Processing CreateTablet for tablet 40b2046b97eb40dea7a35a406566cd0b (DEFAULT_TABLE table=test-table [id=5bc94a862237402186e8177fcc404de0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:20.341777 20826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b2046b97eb40dea7a35a406566cd0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:20.341778 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b2046b97eb40dea7a35a406566cd0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:20.341779 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b2046b97eb40dea7a35a406566cd0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:20.342926 21127 tablet_bootstrap.cc:492] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: Bootstrap starting.
I20260809 16:11:20.343176 21128 tablet_bootstrap.cc:492] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: Bootstrap starting.
I20260809 16:11:20.343405 21129 tablet_bootstrap.cc:492] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: Bootstrap starting.
I20260809 16:11:20.343492 21127 tablet_bootstrap.cc:654] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:20.343492 21128 tablet_bootstrap.cc:654] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:20.343832 21129 tablet_bootstrap.cc:654] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:20.344089 21128 tablet_bootstrap.cc:492] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: No bootstrap required, opened a new log
I20260809 16:11:20.344132 21128 ts_tablet_manager.cc:1397] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:20.344245 21128 raft_consensus.cc:348] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.344304 21128 raft_consensus.cc:374] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:20.344321 21128 raft_consensus.cc:729] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922c00cd71d448f89c6f1af4580b615e, State: Initialized, Role: FOLLOWER
I20260809 16:11:20.344373 21128 consensus_queue.cc:260] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.344550 21128 ts_tablet_manager.cc:1428] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:20.344580 21129 tablet_bootstrap.cc:492] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: No bootstrap required, opened a new log
I20260809 16:11:20.344624 21129 ts_tablet_manager.cc:1397] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:20.344839 21127 tablet_bootstrap.cc:492] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: No bootstrap required, opened a new log
I20260809 16:11:20.344841 21129 raft_consensus.cc:348] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.344894 21127 ts_tablet_manager.cc:1397] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:20.344907 21129 raft_consensus.cc:374] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:20.344951 21129 raft_consensus.cc:729] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54656118cb234510b010a4eef8567eea, State: Initialized, Role: FOLLOWER
I20260809 16:11:20.345016 21129 consensus_queue.cc:260] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.345013 21127 raft_consensus.cc:348] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.345082 21127 raft_consensus.cc:374] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:20.345100 21127 raft_consensus.cc:729] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Initialized, Role: FOLLOWER
I20260809 16:11:20.345183 21129 ts_tablet_manager.cc:1428] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:20.345171 21127 consensus_queue.cc:260] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.345268 21127 ts_tablet_manager.cc:1428] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:20.352597 21136 raft_consensus.cc:482] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:20.352662 21136 raft_consensus.cc:504] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.352787 21136 leader_election.cc:288] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:20.352953 20616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b2046b97eb40dea7a35a406566cd0b" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5263387acd3f43d2a914a90fb331a96d" is_pre_election: true
I20260809 16:11:20.352960 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b2046b97eb40dea7a35a406566cd0b" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea" is_pre_election: true
I20260809 16:11:20.353044 20616 raft_consensus.cc:2454] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 0.
I20260809 16:11:20.353044 20858 raft_consensus.cc:2454] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 0.
I20260809 16:11:20.353175 20258 leader_election.cc:302] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54656118cb234510b010a4eef8567eea, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:20.353267 21136 raft_consensus.cc:2781] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:20.353310 21136 raft_consensus.cc:482] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:20.353323 21136 raft_consensus.cc:3037] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:20.353891 21136 raft_consensus.cc:504] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.354135 21136 leader_election.cc:288] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 election: Requested vote from peers 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:20.354154 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b2046b97eb40dea7a35a406566cd0b" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea"
I20260809 16:11:20.354161 20616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b2046b97eb40dea7a35a406566cd0b" candidate_uuid: "922c00cd71d448f89c6f1af4580b615e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5263387acd3f43d2a914a90fb331a96d"
I20260809 16:11:20.354231 20616 raft_consensus.cc:3037] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:20.354231 20858 raft_consensus.cc:3037] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:20.354818 20616 raft_consensus.cc:2454] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 1.
I20260809 16:11:20.354823 20858 raft_consensus.cc:2454] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922c00cd71d448f89c6f1af4580b615e in term 1.
I20260809 16:11:20.354982 20258 leader_election.cc:302] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54656118cb234510b010a4eef8567eea, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:20.355100 21136 raft_consensus.cc:2781] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:20.355233 21136 raft_consensus.cc:686] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 1 LEADER]: Becoming Leader. State: Replica: 922c00cd71d448f89c6f1af4580b615e, State: Running, Role: LEADER
I20260809 16:11:20.355278 21136 consensus_queue.cc:237] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.355795 20099 catalog_manager.cc:5495] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e reported cstate change: term changed from 0 to 1, leader changed from <none> to 922c00cd71d448f89c6f1af4580b615e (127.16.224.129). New cstate: current_term: 1 leader_uuid: "922c00cd71d448f89c6f1af4580b615e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:20.364209 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:20.362013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40420 (local address 127.16.224.129:33939)
0809 16:11:20.362147 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:20.362149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:20.362157 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:20.362182 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:20.362182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:20.362193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:20.362265 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:20.362416 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:20.362944 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:20.363670 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:20.363879 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:20.363921 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:20.364035 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:20.364060 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:20.364091 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:20.364134 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 16:11:20.364153 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260809 16:11:20.366192 20858 raft_consensus.cc:1261] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Refusing update from remote peer 922c00cd71d448f89c6f1af4580b615e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:20.366209 20616 raft_consensus.cc:1261] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Refusing update from remote peer 922c00cd71d448f89c6f1af4580b615e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:20.366421 21138 consensus_queue.cc:1035] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:20.366540 21136 consensus_queue.cc:1035] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [LEADER]: Connected to new peer: Peer: permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:20.376328 20099 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:20.376416 20099 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:20.376951 20099 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Sending DeleteTablet for 3 replicas of tablet 40b2046b97eb40dea7a35a406566cd0b
I20260809 16:11:20.377183 20826 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b2046b97eb40dea7a35a406566cd0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:20 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:60254
I20260809 16:11:20.377183 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b2046b97eb40dea7a35a406566cd0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:20 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260809 16:11:20.377274 21110 tablet_replica.cc:330] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: stopping tablet replica
I20260809 16:11:20.377312 21110 raft_consensus.cc:2229] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:20.377297 21108 tablet_replica.cc:330] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: stopping tablet replica
I20260809 16:11:20.377351 21110 raft_consensus.cc:2258] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:20.377359 21108 raft_consensus.cc:2229] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:20.377393 21108 raft_consensus.cc:2258] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:20.377417 20345 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b2046b97eb40dea7a35a406566cd0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:20 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:40400
I20260809 16:11:20.377496 21113 tablet_replica.cc:330] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: stopping tablet replica
I20260809 16:11:20.377427 20099 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 16:11:20.377555 21113 raft_consensus.cc:2229] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 1 LEADER]: Raft consensus shutting down.
W20260809 16:11:20.377573 20099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:20.377599 21113 raft_consensus.cc:2258] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:20.377696 21108 ts_tablet_manager.cc:1905] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:20.377822 21110 ts_tablet_manager.cc:1905] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:20.378039 21113 ts_tablet_manager.cc:1905] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:20.378724 20826 tablet_service.cc:1467] Processing CreateTablet for tablet 652095b3b02d4949858407ff38778b5a (DEFAULT_TABLE table=test-table [id=4874eea5c9e64e58936a34a900fd2c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:20.378832 20826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652095b3b02d4949858407ff38778b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:20.379408 21110 ts_tablet_manager.cc:1918] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:20.379465 21110 log.cc:1196] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/40b2046b97eb40dea7a35a406566cd0b
I20260809 16:11:20.379524 21108 ts_tablet_manager.cc:1918] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:20.379573 21108 log.cc:1196] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/40b2046b97eb40dea7a35a406566cd0b
I20260809 16:11:20.379669 21110 ts_tablet_manager.cc:1939] T 40b2046b97eb40dea7a35a406566cd0b P 54656118cb234510b010a4eef8567eea: Deleting consensus metadata
I20260809 16:11:20.379727 21113 ts_tablet_manager.cc:1918] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:20.379768 21113 log.cc:1196] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/40b2046b97eb40dea7a35a406566cd0b
I20260809 16:11:20.379782 21108 ts_tablet_manager.cc:1939] T 40b2046b97eb40dea7a35a406566cd0b P 5263387acd3f43d2a914a90fb331a96d: Deleting consensus metadata
I20260809 16:11:20.379959 21113 ts_tablet_manager.cc:1939] T 40b2046b97eb40dea7a35a406566cd0b P 922c00cd71d448f89c6f1af4580b615e: Deleting consensus metadata
I20260809 16:11:20.379968 20081 catalog_manager.cc:4841] TS 54656118cb234510b010a4eef8567eea (127.16.224.131:44659): tablet 40b2046b97eb40dea7a35a406566cd0b (table test-table [id=5bc94a862237402186e8177fcc404de0]) successfully deleted
I20260809 16:11:20.380055 21129 tablet_bootstrap.cc:492] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: Bootstrap starting.
I20260809 16:11:20.380049 20345 tablet_service.cc:1467] Processing CreateTablet for tablet 652095b3b02d4949858407ff38778b5a (DEFAULT_TABLE table=test-table [id=4874eea5c9e64e58936a34a900fd2c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:20.380139 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652095b3b02d4949858407ff38778b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:20.380252 20081 catalog_manager.cc:4841] TS 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939): tablet 40b2046b97eb40dea7a35a406566cd0b (table test-table [id=5bc94a862237402186e8177fcc404de0]) successfully deleted
I20260809 16:11:20.380321 21129 tablet_bootstrap.cc:654] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:20.380836 21129 tablet_bootstrap.cc:492] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: No bootstrap required, opened a new log
I20260809 16:11:20.380880 21129 ts_tablet_manager.cc:1397] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:20.381026 21128 tablet_bootstrap.cc:492] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e: Bootstrap starting.
I20260809 16:11:20.381021 21129 raft_consensus.cc:348] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.381084 21129 raft_consensus.cc:374] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:20.381104 21129 raft_consensus.cc:729] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54656118cb234510b010a4eef8567eea, State: Initialized, Role: FOLLOWER
I20260809 16:11:20.381146 21129 consensus_queue.cc:260] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.381228 21129 ts_tablet_manager.cc:1428] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:20.381345 21128 tablet_bootstrap.cc:654] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:20.381374 20082 catalog_manager.cc:4841] TS 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341): tablet 40b2046b97eb40dea7a35a406566cd0b (table test-table [id=5bc94a862237402186e8177fcc404de0]) successfully deleted
I20260809 16:11:20.381458 20580 tablet_service.cc:1467] Processing CreateTablet for tablet 652095b3b02d4949858407ff38778b5a (DEFAULT_TABLE table=test-table [id=4874eea5c9e64e58936a34a900fd2c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:20.381549 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652095b3b02d4949858407ff38778b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:20.381834 21128 tablet_bootstrap.cc:492] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e: No bootstrap required, opened a new log
I20260809 16:11:20.381887 21128 ts_tablet_manager.cc:1397] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:20.381989 21128 raft_consensus.cc:348] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.382055 21128 raft_consensus.cc:374] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:20.382081 21128 raft_consensus.cc:729] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922c00cd71d448f89c6f1af4580b615e, State: Initialized, Role: FOLLOWER
I20260809 16:11:20.382126 21128 consensus_queue.cc:260] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.382218 21128 ts_tablet_manager.cc:1428] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:20.382740 21127 tablet_bootstrap.cc:492] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: Bootstrap starting.
I20260809 16:11:20.383047 21127 tablet_bootstrap.cc:654] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:20.383543 21127 tablet_bootstrap.cc:492] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: No bootstrap required, opened a new log
I20260809 16:11:20.383594 21127 ts_tablet_manager.cc:1397] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:11:20.383690 21127 raft_consensus.cc:348] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.383740 21127 raft_consensus.cc:374] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:20.383752 21127 raft_consensus.cc:729] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Initialized, Role: FOLLOWER
I20260809 16:11:20.383783 21127 consensus_queue.cc:260] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.383868 21127 ts_tablet_manager.cc:1428] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:20.599793 21028 raft_consensus.cc:482] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:20.599912 21028 raft_consensus.cc:504] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.600091 21028 leader_election.cc:288] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:20.600421 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652095b3b02d4949858407ff38778b5a" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c00cd71d448f89c6f1af4580b615e" is_pre_election: true
I20260809 16:11:20.600421 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652095b3b02d4949858407ff38778b5a" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea" is_pre_election: true
I20260809 16:11:20.600553 20391 raft_consensus.cc:2454] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 0.
I20260809 16:11:20.600551 20858 raft_consensus.cc:2454] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 0.
I20260809 16:11:20.600741 20504 leader_election.cc:302] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:20.600848 21028 raft_consensus.cc:2781] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:20.600893 21028 raft_consensus.cc:482] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:20.600917 21028 raft_consensus.cc:3037] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:20.601594 21028 raft_consensus.cc:504] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.601735 21028 leader_election.cc:288] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 election: Requested vote from peers 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939), 54656118cb234510b010a4eef8567eea (127.16.224.131:44659)
I20260809 16:11:20.601943 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652095b3b02d4949858407ff38778b5a" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c00cd71d448f89c6f1af4580b615e"
I20260809 16:11:20.602038 20391 raft_consensus.cc:3037] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:20.602031 20858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652095b3b02d4949858407ff38778b5a" candidate_uuid: "5263387acd3f43d2a914a90fb331a96d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54656118cb234510b010a4eef8567eea"
I20260809 16:11:20.602145 20858 raft_consensus.cc:3037] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:20.602900 20391 raft_consensus.cc:2454] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 1.
I20260809 16:11:20.603031 20858 raft_consensus.cc:2454] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5263387acd3f43d2a914a90fb331a96d in term 1.
I20260809 16:11:20.603102 20504 leader_election.cc:302] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5263387acd3f43d2a914a90fb331a96d, 922c00cd71d448f89c6f1af4580b615e; no voters:
I20260809 16:11:20.603216 21028 raft_consensus.cc:2781] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:20.603276 21028 raft_consensus.cc:686] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 1 LEADER]: Becoming Leader. State: Replica: 5263387acd3f43d2a914a90fb331a96d, State: Running, Role: LEADER
I20260809 16:11:20.603353 21028 consensus_queue.cc:237] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } }
I20260809 16:11:20.603978 20099 catalog_manager.cc:5495] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5263387acd3f43d2a914a90fb331a96d (127.16.224.130). New cstate: current_term: 1 leader_uuid: "5263387acd3f43d2a914a90fb331a96d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5263387acd3f43d2a914a90fb331a96d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:20.614771 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:20.612314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48096 (local address 127.16.224.130:40341)
0809 16:11:20.612504 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:20.612506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:20.612514 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:20.612537 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:20.612538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:20.612550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:20.612619 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:20.612716 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:20.613460 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:20.614188 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:20.614455 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:20.614510 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:20.614609 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:20.614636 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:20.614666 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:20.614697 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 16:11:20.614713 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":118,"threads_started":1}
I20260809 16:11:20.616745 20391 raft_consensus.cc:1261] T 652095b3b02d4949858407ff38778b5a P 922c00cd71d448f89c6f1af4580b615e [term 1 FOLLOWER]: Refusing update from remote peer 5263387acd3f43d2a914a90fb331a96d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:20.616753 20858 raft_consensus.cc:1261] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Refusing update from remote peer 5263387acd3f43d2a914a90fb331a96d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:20.616935 21083 consensus_queue.cc:1035] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Connected to new peer: Peer: permanent_uuid: "54656118cb234510b010a4eef8567eea" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:20.617074 21076 consensus_queue.cc:1035] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [LEADER]: Connected to new peer: Peer: permanent_uuid: "922c00cd71d448f89c6f1af4580b615e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:11:20.620963 21074 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:11:22.137614 20103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:22.137724 20103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:22.138625 20103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d5962fb43e549f2a7a3b878eee5d849: Sending DeleteTablet for 3 replicas of tablet 652095b3b02d4949858407ff38778b5a
I20260809 16:11:22.138911 20345 tablet_service.cc:1514] Processing DeleteTablet for tablet 652095b3b02d4949858407ff38778b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:22 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:40400
I20260809 16:11:22.139037 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20231
I20260809 16:11:22.139106 20580 tablet_service.cc:1514] Processing DeleteTablet for tablet 652095b3b02d4949858407ff38778b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:22 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260809 16:11:22.139501 20826 tablet_service.cc:1514] Processing DeleteTablet for tablet 652095b3b02d4949858407ff38778b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:22 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:60254
I20260809 16:11:22.154529 21199 tablet_replica.cc:330] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: stopping tablet replica
I20260809 16:11:22.154529 21198 tablet_replica.cc:330] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: stopping tablet replica
I20260809 16:11:22.154613 21198 raft_consensus.cc:2229] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:22.154613 21199 raft_consensus.cc:2229] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:22.154664 21199 raft_consensus.cc:2258] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:22.154743 21198 raft_consensus.cc:2258] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:22.154928 21199 ts_tablet_manager.cc:1905] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:22.155045 21198 ts_tablet_manager.cc:1905] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:22.156639 21199 ts_tablet_manager.cc:1918] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:11:22.156703 21198 ts_tablet_manager.cc:1918] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:11:22.156736 21199 log.cc:1196] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/652095b3b02d4949858407ff38778b5a
I20260809 16:11:22.156795 21198 log.cc:1196] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/652095b3b02d4949858407ff38778b5a
I20260809 16:11:22.156991 21199 ts_tablet_manager.cc:1939] T 652095b3b02d4949858407ff38778b5a P 54656118cb234510b010a4eef8567eea: Deleting consensus metadata
I20260809 16:11:22.157055 21198 ts_tablet_manager.cc:1939] T 652095b3b02d4949858407ff38778b5a P 5263387acd3f43d2a914a90fb331a96d: Deleting consensus metadata
I20260809 16:11:22.157471 20082 catalog_manager.cc:4841] TS 5263387acd3f43d2a914a90fb331a96d (127.16.224.130:40341): tablet 652095b3b02d4949858407ff38778b5a (table test-table [id=4874eea5c9e64e58936a34a900fd2c87]) successfully deleted
I20260809 16:11:22.157541 20081 catalog_manager.cc:4841] TS 54656118cb234510b010a4eef8567eea (127.16.224.131:44659): tablet 652095b3b02d4949858407ff38778b5a (table test-table [id=4874eea5c9e64e58936a34a900fd2c87]) successfully deleted
W20260809 16:11:22.160635 20081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:33939 (error 108)
I20260809 16:11:22.160869 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20478
I20260809 16:11:22.168243 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:22.160994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:33939 (local address 127.0.0.1:40422)
0809 16:11:22.168145 (+ 7151us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:22.168204 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:33939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7122,"thread_start_us":56,"threads_started":1}
W20260809 16:11:22.168350 20081 catalog_manager.cc:4584] TS 922c00cd71d448f89c6f1af4580b615e (127.16.224.129:33939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 652095b3b02d4949858407ff38778b5a: Network error: Client connection negotiation failed: client connection to 127.16.224.129:33939: connect: Connection refused (error 111)
I20260809 16:11:22.178972 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20712
I20260809 16:11:22.189360 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20054
2026-08-09T16:11:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6765 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:22 dist-test-slave-fgb3 krb5kdc[21206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:22 dist-test-slave-fgb3 krb5kdc[21206](info): set up 2 sockets
Aug 09 16:11:22 dist-test-slave-fgb3 krb5kdc[21206](info): commencing operation
krb5kdc: starting...
W20260809 16:11:23.328228 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:23 dist-test-slave-fgb3 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:23.440929 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44219
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:44219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:23.523063 21225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:23.523277 21225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:23.523298 21225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:23.524430 21225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:23.524477 21225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:23.524489 21225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:23.524500 21225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:23.524511 21225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:23.526041 21225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:44219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:23.526285 21225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:23.526542 21225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:23.531417 21234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:23.531471 21238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:23.531517 21225 server_base.cc:1034] running on GCE node
W20260809 16:11:23.531418 21235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:23.531833 21225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:23.532109 21225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:23.533245 21225 hybrid_clock.cc:648] HybridClock initialized: now 1786291883533227 us; error 35 us; skew 500 ppm
Aug 09 16:11:23 dist-test-slave-fgb3 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291883, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:23.535773 21225 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:23.536193 21225 webserver.cc:460] Webserver started at http://127.16.224.190:37431/ using document root <none> and password file <none>
I20260809 16:11:23.536391 21225 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:23.536438 21225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:23.536561 21225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:23.537602 21225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "8a834099c10842b6ba1b63e434a0a9c6"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "4d707ccf7fc36e2fcdd8202ce6bd6b7c"
server_key_iv: "54aed1e993727db394a0e92c709d2abc"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.537946 21225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "8a834099c10842b6ba1b63e434a0a9c6"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "4d707ccf7fc36e2fcdd8202ce6bd6b7c"
server_key_iv: "54aed1e993727db394a0e92c709d2abc"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.539294 21225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:23.539951 21247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.540110 21225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:23.540196 21225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "8a834099c10842b6ba1b63e434a0a9c6"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "4d707ccf7fc36e2fcdd8202ce6bd6b7c"
server_key_iv: "54aed1e993727db394a0e92c709d2abc"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.540258 21225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:23.555991 21225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:23.558158 21225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:23.558305 21225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:23.562358 21225 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:44219
I20260809 16:11:23.562409 21344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:44219 every 8 connection(s)
I20260809 16:11:23.562757 21225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:23.563717 21347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:23.566125 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 21225
I20260809 16:11:23.566211 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:23.566197 21347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6: Bootstrap starting.
I20260809 16:11:23.566412 17282 external_mini_cluster.cc:1428] Setting key 675a56e555e94405e7f20a06cc974156
I20260809 16:11:23.566951 21347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:23.567301 21347 log.cc:824] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:23.568086 21347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6: No bootstrap required, opened a new log
I20260809 16:11:23.569981 21347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a834099c10842b6ba1b63e434a0a9c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44219 } }
I20260809 16:11:23.570130 21347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:23.570166 21347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a834099c10842b6ba1b63e434a0a9c6, State: Initialized, Role: FOLLOWER
I20260809 16:11:23.570341 21347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a834099c10842b6ba1b63e434a0a9c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44219 } }
I20260809 16:11:23.570443 21347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:23.570477 21347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:23.570524 21347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:23.571269 21347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a834099c10842b6ba1b63e434a0a9c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44219 } }
I20260809 16:11:23.571372 21347 leader_election.cc:302] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a834099c10842b6ba1b63e434a0a9c6; no voters:
I20260809 16:11:23.571549 21347 leader_election.cc:288] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:23.571590 21359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:23.571719 21359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [term 1 LEADER]: Becoming Leader. State: Replica: 8a834099c10842b6ba1b63e434a0a9c6, State: Running, Role: LEADER
I20260809 16:11:23.571838 21347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:23.571892 21359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a834099c10842b6ba1b63e434a0a9c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44219 } }
I20260809 16:11:23.572306 21359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a834099c10842b6ba1b63e434a0a9c6. Latest consensus state: current_term: 1 leader_uuid: "8a834099c10842b6ba1b63e434a0a9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a834099c10842b6ba1b63e434a0a9c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44219 } } }
I20260809 16:11:23.572357 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a834099c10842b6ba1b63e434a0a9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a834099c10842b6ba1b63e434a0a9c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44219 } } }
I20260809 16:11:23.572491 21359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:23.572501 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:23.572785 21375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 16:11:23 dist-test-slave-fgb3 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:23.573364 21375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:23.575323 21375 catalog_manager.cc:1347] Generated new cluster ID: ce3b11dc81c24a31b655d5f483ed5aba
I20260809 16:11:23.575383 21375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:23.577409 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:23.566917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.16.224.190:44219)
0809 16:11:23.567160 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:23.567167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:23.567194 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:11:23.567736 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:23.567747 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:23.567816 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:23.567958 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:23.568618 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.569529 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.570276 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.570923 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.571058 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:23.573779 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:23.573783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:23.573784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:23.573797 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:23.575736 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:23.576119 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:23.576122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:23.576122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:23.576233 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:23.576412 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:23.576413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:23.576413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:23.576697 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:23.576809 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:23.577067 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 16:11:23.577206 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":72,"threads_started":1}
I20260809 16:11:23.585435 21375 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:23.586155 21375 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:23.596856 21375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a834099c10842b6ba1b63e434a0a9c6: Generated new TSK 0
I20260809 16:11:23.597059 21375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:23.622993 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44219
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:23.705482 21393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:23.705683 21393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:23.705706 21393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:23.706919 21393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:23.706965 21393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:23.706992 21393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:23.708542 21393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:23.708786 21393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:23.709014 21393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:23.713882 21399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:23.713940 21398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:23.713881 21402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:23.714118 21393 server_base.cc:1034] running on GCE node
I20260809 16:11:23.714392 21393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:23.714639 21393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:23.715832 21393 hybrid_clock.cc:648] HybridClock initialized: now 1786291883715812 us; error 32 us; skew 500 ppm
Aug 09 16:11:23 dist-test-slave-fgb3 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291883, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:23.718585 21393 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:23.719012 21393 webserver.cc:460] Webserver started at http://127.16.224.129:40889/ using document root <none> and password file <none>
I20260809 16:11:23.719214 21393 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:23.719267 21393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:23.719370 21393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:23.720441 21393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "86111b70151c45898a11f7a480f4b017"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "6a52a61cd2ef7b999e994a3a8c21062d"
server_key_iv: "f8343e8199b4c5b720586fdd264a9f7d"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.720775 21393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "86111b70151c45898a11f7a480f4b017"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "6a52a61cd2ef7b999e994a3a8c21062d"
server_key_iv: "f8343e8199b4c5b720586fdd264a9f7d"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.722097 21393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:23.722932 21412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.723088 21393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.723182 21393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "86111b70151c45898a11f7a480f4b017"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "6a52a61cd2ef7b999e994a3a8c21062d"
server_key_iv: "f8343e8199b4c5b720586fdd264a9f7d"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.723275 21393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:23.733284 21393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:23.735040 21393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:23.735193 21393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:23.735416 21393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:23.735723 21393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:23.735757 21393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.735786 21393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:23.735802 21393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.741416 21393 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:41325
I20260809 16:11:23.741464 21615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:41325 every 8 connection(s)
I20260809 16:11:23.741775 21393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:11:23 dist-test-slave-fgb3 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291883, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:23.748631 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 21393
I20260809 16:11:23.748733 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:23.748903 17282 external_mini_cluster.cc:1428] Setting key 40788c36f8c551b3b4b36010a60b2c07
I20260809 16:11:23.750753 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:23.742520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:36983 (local address 127.16.224.190:44219)
0809 16:11:23.742603 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:23.742605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:23.742784 (+ 179us) server_negotiation.cc:407] Connection header received
0809 16:11:23.743307 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:23.743308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:23.743317 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:23.743372 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:23.744321 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.744820 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.745757 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.745957 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.745999 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:23.748112 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:23.748113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:23.748114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:23.748121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:23.749433 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:23.749842 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:23.749844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:23.749844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:23.749906 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:23.750121 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:23.750122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:23.750122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:23.750252 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:23.750337 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:23.750583 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 16:11:23.750669 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 16:11:23.750846 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:23.742496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44219 (local address 127.16.224.129:36983)
0809 16:11:23.742658 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:23.742702 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:23.743216 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:23.743439 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:23.743442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:23.744050 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:23.744248 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:23.744253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.744886 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.744887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:23.745695 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:23.745697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.746011 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.746011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:23.746115 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:23.746637 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:23.746646 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:23.748016 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:23.749526 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:23.749535 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:23.749538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:23.749774 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:23.749978 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:23.749979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:23.749979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:23.750072 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:23.750305 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:23.750308 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:23.750461 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 16:11:23.750545 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:11:23.750673 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260809 16:11:23.751400 21616 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44219
I20260809 16:11:23.751510 21616 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:23.751683 21616 heartbeater.cc:511] Master 127.16.224.190:44219 requested a full tablet report, sending...
I20260809 16:11:23.752187 21281 ts_manager.cc:194] Registered new tserver with Master: 86111b70151c45898a11f7a480f4b017 (127.16.224.129:41325)
I20260809 16:11:23.752797 21281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:36983
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:23.772536 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44219
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:23.855706 21623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:23.855911 21623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:23.855933 21623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:23.857070 21623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:23.857112 21623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:23.857136 21623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:23.858605 21623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:23.858834 21623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:23.859042 21623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:23.863932 21633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:23.863996 21623 server_base.cc:1034] running on GCE node
W20260809 16:11:23.863963 21632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:23.863957 21639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:23.864239 21623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:23.864483 21623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:23.865630 21623 hybrid_clock.cc:648] HybridClock initialized: now 1786291883865614 us; error 28 us; skew 500 ppm
Aug 09 16:11:23 dist-test-slave-fgb3 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291883, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:23.868083 21623 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:23.868486 21623 webserver.cc:460] Webserver started at http://127.16.224.130:38399/ using document root <none> and password file <none>
I20260809 16:11:23.868681 21623 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:23.868728 21623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:23.868862 21623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:23.869895 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "f8d8c082361e4127b31c4edd6c8cb539"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "792492eff05bd2d1074ca98c26ae62fd"
server_key_iv: "834685415071927290b36fbafd74c0eb"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.870237 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "f8d8c082361e4127b31c4edd6c8cb539"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "792492eff05bd2d1074ca98c26ae62fd"
server_key_iv: "834685415071927290b36fbafd74c0eb"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.871551 21623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:23.872291 21647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.872433 21623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:23.872524 21623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "f8d8c082361e4127b31c4edd6c8cb539"
format_stamp: "Formatted at 2026-08-09 16:11:23 on dist-test-slave-fgb3"
server_key: "792492eff05bd2d1074ca98c26ae62fd"
server_key_iv: "834685415071927290b36fbafd74c0eb"
server_key_version: "encryptionkey@0"
I20260809 16:11:23.872592 21623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:23.904582 21623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:23.906860 21623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:23.907006 21623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:23.907245 21623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:23.907560 21623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:23.907593 21623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.907622 21623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:23.907646 21623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:23.913167 21623 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:33469
I20260809 16:11:23.913216 21847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:33469 every 8 connection(s)
I20260809 16:11:23.913529 21623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:23.918036 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 21623
I20260809 16:11:23.918128 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:23.918365 17282 external_mini_cluster.cc:1428] Setting key 530eb8c5da71f8fb2d6683a60c8448d7
Aug 09 16:11:23 dist-test-slave-fgb3 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291883, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:23.922711 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:23.914193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:57589 (local address 127.16.224.190:44219)
0809 16:11:23.914236 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:23.914238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:23.914463 (+ 225us) server_negotiation.cc:407] Connection header received
0809 16:11:23.914927 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:23.914928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:23.914939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:23.914993 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:23.915903 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.916407 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.917389 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.917581 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.917632 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:23.919990 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:23.919992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:23.919992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:23.920000 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:23.921324 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:23.921714 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:23.921715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:23.921716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:23.921783 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:23.922017 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:23.922018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:23.922018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:23.922131 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:23.922196 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:23.922531 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 16:11:23.922617 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:11:23.922858 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:23.914171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44219 (local address 127.16.224.130:57589)
0809 16:11:23.914329 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:23.914376 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:23.914848 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:23.915056 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:23.915059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:23.915610 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:23.915811 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:23.915817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.916478 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.916479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:23.917309 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:23.917311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:23.917639 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:23.917639 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:23.917732 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:23.918286 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:23.918297 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:23.919891 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:23.921408 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:23.921416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:23.921419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:23.921641 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:23.921844 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:23.921845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:23.921845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:23.921956 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:23.922192 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:23.922199 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:23.922375 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 16:11:23.922482 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 16:11:23.922640 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260809 16:11:23.923312 21849 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44219
I20260809 16:11:23.923405 21849 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:23.923646 21849 heartbeater.cc:511] Master 127.16.224.190:44219 requested a full tablet report, sending...
I20260809 16:11:23.924024 21281 ts_manager.cc:194] Registered new tserver with Master: f8d8c082361e4127b31c4edd6c8cb539 (127.16.224.130:33469)
I20260809 16:11:23.924407 21281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:57589
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:23.941063 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44219
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:24.022217 21854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:24.022455 21854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:24.022487 21854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:24.023638 21854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:24.023679 21854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:24.023711 21854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:24.025166 21854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:24.025426 21854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:24.025652 21854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:24.030488 21862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:24.030481 21860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:24.030639 21854 server_base.cc:1034] running on GCE node
W20260809 16:11:24.030481 21866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:24.030884 21854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:24.031147 21854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:24.032284 21854 hybrid_clock.cc:648] HybridClock initialized: now 1786291884032269 us; error 29 us; skew 500 ppm
Aug 09 16:11:24 dist-test-slave-fgb3 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291884, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:24.034721 21854 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:24.035180 21854 webserver.cc:460] Webserver started at http://127.16.224.131:38805/ using document root <none> and password file <none>
I20260809 16:11:24.035419 21854 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:24.035473 21854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:24.035606 21854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:24.036664 21854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "2f29fd0dacd34174b7f0f75682a76fc7"
format_stamp: "Formatted at 2026-08-09 16:11:24 on dist-test-slave-fgb3"
server_key: "661482ed4d9c8764a193ab67f75400c9"
server_key_iv: "085259a70fad115c7212d9875e87d196"
server_key_version: "encryptionkey@0"
I20260809 16:11:24.037019 21854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "2f29fd0dacd34174b7f0f75682a76fc7"
format_stamp: "Formatted at 2026-08-09 16:11:24 on dist-test-slave-fgb3"
server_key: "661482ed4d9c8764a193ab67f75400c9"
server_key_iv: "085259a70fad115c7212d9875e87d196"
server_key_version: "encryptionkey@0"
I20260809 16:11:24.038278 21854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:11:24.039002 21878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:24.039132 21854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:24.039222 21854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "2f29fd0dacd34174b7f0f75682a76fc7"
format_stamp: "Formatted at 2026-08-09 16:11:24 on dist-test-slave-fgb3"
server_key: "661482ed4d9c8764a193ab67f75400c9"
server_key_iv: "085259a70fad115c7212d9875e87d196"
server_key_version: "encryptionkey@0"
I20260809 16:11:24.039296 21854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:24.052040 21854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:24.053673 21854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:24.053798 21854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:24.054008 21854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:24.054296 21854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:24.054358 21854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:24.054395 21854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:24.054419 21854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:24.060081 21854 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:35393
I20260809 16:11:24.060137 22077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:35393 every 8 connection(s)
I20260809 16:11:24.060427 21854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:24.065982 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 21854
I20260809 16:11:24.066090 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
Aug 09 16:11:24 dist-test-slave-fgb3 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291884, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:24.066286 17282 external_mini_cluster.cc:1428] Setting key 4c3ea8c767b6ad4e8bb9814ddd7e2ae3
I20260809 16:11:24.069057 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:24.061144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:36033 (local address 127.16.224.190:44219)
0809 16:11:24.061191 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:24.061193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:24.061372 (+ 179us) server_negotiation.cc:407] Connection header received
0809 16:11:24.061861 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:24.061863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:24.061872 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:24.061925 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:24.062838 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:24.063333 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:24.064256 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:24.064428 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:24.064471 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:24.066599 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:24.066601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:24.066601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:24.066608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:24.068000 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:24.068340 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:24.068340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:24.068340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:24.068388 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:24.068558 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:24.068558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:24.068559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:24.068659 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:24.068708 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:24.068909 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:11:24.068992 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:24.069209 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:24.061099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44219 (local address 127.16.224.131:36033)
0809 16:11:24.061258 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:24.061300 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:24.061780 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:24.061983 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:24.061987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:24.062573 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:24.062763 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:24.062769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:24.063393 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:24.063394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:24.064197 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:24.064199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:24.064496 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:24.064497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:24.064591 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:24.065046 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:24.065054 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:24.066515 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:24.068066 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:24.068073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:24.068075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:24.068284 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:24.068430 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:24.068430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:24.068431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:24.068518 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:24.068699 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:24.068702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:24.068816 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 16:11:24.068886 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:11:24.069011 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260809 16:11:24.069644 22078 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44219
I20260809 16:11:24.069737 22078 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:24.069911 22078 heartbeater.cc:511] Master 127.16.224.190:44219 requested a full tablet report, sending...
I20260809 16:11:24.070258 21281 ts_manager.cc:194] Registered new tserver with Master: 2f29fd0dacd34174b7f0f75682a76fc7 (127.16.224.131:35393)
I20260809 16:11:24.070655 21281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:36033
I20260809 16:11:24.077972 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:11:24.089643 21353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:24.088783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.16.224.190:44219)
0809 16:11:24.088854 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:24.088856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:24.088920 (+ 64us) server_negotiation.cc:407] Connection header received
0809 16:11:24.088985 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:24.088986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:24.088995 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:24.089043 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:24.089574 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55448: BlockingRecv error: recv got EOF from 127.0.0.1:55448 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:11:24.091245 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 21393
I20260809 16:11:24.108156 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 21623
I20260809 16:11:24.121227 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 21854
I20260809 16:11:24.129877 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 21225
2026-08-09T16:11:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1938 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:24 dist-test-slave-fgb3 krb5kdc[22097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:24 dist-test-slave-fgb3 krb5kdc[22097](info): set up 2 sockets
Aug 09 16:11:24 dist-test-slave-fgb3 krb5kdc[22097](info): commencing operation
krb5kdc: starting...
W20260809 16:11:25.270411 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:25 dist-test-slave-fgb3 krb5kdc[22097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:25.386989 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44141
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:44141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:25.477334 22129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:25.477707 22129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:25.477743 22129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:25.479631 22129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:25.479713 22129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:25.479749 22129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:25.479789 22129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:25.479841 22129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:25.482405 22129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:44141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:25.482854 22129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:25.483280 22129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:25.490731 22137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:25.490993 22129 server_base.cc:1034] running on GCE node
W20260809 16:11:25.490736 22135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:25.490757 22139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:25.491536 22129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:25.491978 22129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:25.493146 22129 hybrid_clock.cc:648] HybridClock initialized: now 1786291885493124 us; error 50 us; skew 500 ppm
Aug 09 16:11:25 dist-test-slave-fgb3 krb5kdc[22097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291885, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:25.499662 22129 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:25.500469 22129 webserver.cc:460] Webserver started at http://127.16.224.190:33777/ using document root <none> and password file <none>
I20260809 16:11:25.500897 22129 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:25.500976 22129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:25.501243 22129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:25.503039 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "2baeb735f7d74934896c9e4dc7c6033b"
format_stamp: "Formatted at 2026-08-09 16:11:25 on dist-test-slave-fgb3"
server_key: "0a339da1ac08a615c177ba318c75d569"
server_key_iv: "d6afec454e5e825d684f87c1f0c7763d"
server_key_version: "encryptionkey@0"
I20260809 16:11:25.503608 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "2baeb735f7d74934896c9e4dc7c6033b"
format_stamp: "Formatted at 2026-08-09 16:11:25 on dist-test-slave-fgb3"
server_key: "0a339da1ac08a615c177ba318c75d569"
server_key_iv: "d6afec454e5e825d684f87c1f0c7763d"
server_key_version: "encryptionkey@0"
I20260809 16:11:25.505677 22129 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:11:25.506834 22150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:25.507050 22129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:11:25.507212 22129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "2baeb735f7d74934896c9e4dc7c6033b"
format_stamp: "Formatted at 2026-08-09 16:11:25 on dist-test-slave-fgb3"
server_key: "0a339da1ac08a615c177ba318c75d569"
server_key_iv: "d6afec454e5e825d684f87c1f0c7763d"
server_key_version: "encryptionkey@0"
I20260809 16:11:25.507331 22129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:25.540877 22129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:25.544731 22129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:25.544952 22129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:25.551685 22129 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:44141
I20260809 16:11:25.551760 22236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:44141 every 8 connection(s)
I20260809 16:11:25.552510 22129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:25.552976 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 22129
I20260809 16:11:25.553115 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:25.553341 17282 external_mini_cluster.cc:1428] Setting key 2019b78b86228c3feb5d901ba65fff43
I20260809 16:11:25.553735 22237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:25.558511 22237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b: Bootstrap starting.
I20260809 16:11:25.559530 22237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:25.560021 22237 log.cc:824] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:25.561414 22237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b: No bootstrap required, opened a new log
Aug 09 16:11:25 dist-test-slave-fgb3 krb5kdc[22097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:25.564787 22237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baeb735f7d74934896c9e4dc7c6033b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44141 } }
I20260809 16:11:25.565037 22237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:25.565110 22237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2baeb735f7d74934896c9e4dc7c6033b, State: Initialized, Role: FOLLOWER
I20260809 16:11:25.565351 22237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baeb735f7d74934896c9e4dc7c6033b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44141 } }
I20260809 16:11:25.565454 22237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:25.565492 22237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:25.565572 22237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:25.566532 22237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baeb735f7d74934896c9e4dc7c6033b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44141 } }
I20260809 16:11:25.566715 22237 leader_election.cc:302] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2baeb735f7d74934896c9e4dc7c6033b; no voters:
I20260809 16:11:25.567020 22237 leader_election.cc:288] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:25.567026 22244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:25.567265 22244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [term 1 LEADER]: Becoming Leader. State: Replica: 2baeb735f7d74934896c9e4dc7c6033b, State: Running, Role: LEADER
I20260809 16:11:25.567476 22244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baeb735f7d74934896c9e4dc7c6033b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44141 } }
I20260809 16:11:25.567512 22237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:25.568142 22246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2baeb735f7d74934896c9e4dc7c6033b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baeb735f7d74934896c9e4dc7c6033b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44141 } } }
I20260809 16:11:25.568917 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:25.554049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.16.224.190:44141)
0809 16:11:25.554405 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:25.554424 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:25.554469 (+ 45us) server_negotiation.cc:407] Connection header received
0809 16:11:25.555176 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:25.555185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:25.555263 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:25.555476 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:25.556428 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:25.557783 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:25.558922 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:25.559570 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:25.559746 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:25.563874 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:25.563879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:25.563879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:25.563900 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:25.566373 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:25.566972 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:25.566985 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:25.566985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:25.567121 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:25.567459 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:25.567460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:25.567460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:25.567941 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:25.568166 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:25.568355 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 16:11:25.568596 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":173,"threads_started":1}
I20260809 16:11:25.568819 22245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2baeb735f7d74934896c9e4dc7c6033b. Latest consensus state: current_term: 1 leader_uuid: "2baeb735f7d74934896c9e4dc7c6033b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baeb735f7d74934896c9e4dc7c6033b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44141 } } }
I20260809 16:11:25.569651 22245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:25.568960 22246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:25.570734 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 16:11:25.571732 22275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:11:25.571825 22275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:11:25.571915 22277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:25.572170 22277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:25.574640 22277 catalog_manager.cc:1347] Generated new cluster ID: efe41297bebe44b1959aa13541814313
I20260809 16:11:25.574729 22277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:25.585981 22277 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:25.586745 22277 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:25.596603 22277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2baeb735f7d74934896c9e4dc7c6033b: Generated new TSK 0
I20260809 16:11:25.596923 22277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:11:25.606137 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:25.600325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.16.224.190:44141)
0809 16:11:25.600388 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:25.600400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:25.600589 (+ 189us) server_negotiation.cc:407] Connection header received
0809 16:11:25.600744 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:25.600754 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:25.600770 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:25.600882 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:25.601688 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:25.602621 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:25.604192 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:25.604465 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:25.604544 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:25.605543 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:25.605545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:25.605545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:25.605806 (+ 261us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:25.605842 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:25.605850 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:25.605915 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:25.606026 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 16:11:25.606055 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:11:25.610637 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 22129
2026-08-09T16:11:25Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1487 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:25 dist-test-slave-fgb3 krb5kdc[22297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:25 dist-test-slave-fgb3 krb5kdc[22297](info): set up 2 sockets
Aug 09 16:11:25 dist-test-slave-fgb3 krb5kdc[22297](info): commencing operation
krb5kdc: starting...
W20260809 16:11:26.932789 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:27.046043 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43799
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:27.128114 22316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:27.128329 22316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:27.128350 22316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:27.129452 22316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:27.129487 22316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:27.129498 22316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:27.129509 22316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:27.129519 22316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:27.131034 22316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:43799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:27.131246 22316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:27.131451 22316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:27.136139 22326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:27.136113 22325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:27.136396 22316 server_base.cc:1034] running on GCE node
W20260809 16:11:27.136133 22331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:27.136663 22316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:27.136955 22316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:27.138101 22316 hybrid_clock.cc:648] HybridClock initialized: now 1786291887138081 us; error 34 us; skew 500 ppm
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:27.140614 22316 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:27.141064 22316 webserver.cc:460] Webserver started at http://127.16.224.190:36131/ using document root <none> and password file <none>
I20260809 16:11:27.141265 22316 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:27.141314 22316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:27.141463 22316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:27.142586 22316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "a03ee53cde354ae587586729bce06cff"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "ff419cf982f11701fd905153faaf9120"
server_key_iv: "9fb6c7ade23c937691cd0b3181ae2f4b"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.142927 22316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "a03ee53cde354ae587586729bce06cff"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "ff419cf982f11701fd905153faaf9120"
server_key_iv: "9fb6c7ade23c937691cd0b3181ae2f4b"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.144156 22316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:27.144827 22341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.144989 22316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:27.145080 22316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "a03ee53cde354ae587586729bce06cff"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "ff419cf982f11701fd905153faaf9120"
server_key_iv: "9fb6c7ade23c937691cd0b3181ae2f4b"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.145140 22316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:27.161531 22316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:27.163772 22316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:27.163918 22316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:27.167891 22316 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:43799
I20260809 16:11:27.167932 22425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:43799 every 8 connection(s)
I20260809 16:11:27.168263 22316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:27.169386 22426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:27.171104 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 22316
I20260809 16:11:27.171198 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:27.171372 17282 external_mini_cluster.cc:1428] Setting key d56bb6d3a8db3d2bd7ba7b79d085bb0a
I20260809 16:11:27.172003 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff: Bootstrap starting.
I20260809 16:11:27.172731 22426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:27.173058 22426 log.cc:824] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:27.174697 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff: No bootstrap required, opened a new log
I20260809 16:11:27.176789 22426 raft_consensus.cc:348] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a03ee53cde354ae587586729bce06cff" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43799 } }
I20260809 16:11:27.176942 22426 raft_consensus.cc:374] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:27.176963 22426 raft_consensus.cc:729] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03ee53cde354ae587586729bce06cff, State: Initialized, Role: FOLLOWER
I20260809 16:11:27.177083 22426 consensus_queue.cc:260] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a03ee53cde354ae587586729bce06cff" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43799 } }
I20260809 16:11:27.177160 22426 raft_consensus.cc:388] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:27.177183 22426 raft_consensus.cc:482] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:27.177228 22426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 0 FOLLOWER]: Advancing to term 1
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:27.177953 22426 raft_consensus.cc:504] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a03ee53cde354ae587586729bce06cff" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43799 } }
I20260809 16:11:27.178072 22426 leader_election.cc:302] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a03ee53cde354ae587586729bce06cff; no voters:
I20260809 16:11:27.178238 22426 leader_election.cc:288] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:27.178283 22433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:27.178483 22433 raft_consensus.cc:686] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [term 1 LEADER]: Becoming Leader. State: Replica: a03ee53cde354ae587586729bce06cff, State: Running, Role: LEADER
I20260809 16:11:27.178599 22426 sys_catalog.cc:564] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:27.178610 22433 consensus_queue.cc:237] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a03ee53cde354ae587586729bce06cff" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43799 } }
I20260809 16:11:27.179297 22437 sys_catalog.cc:455] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a03ee53cde354ae587586729bce06cff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a03ee53cde354ae587586729bce06cff" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43799 } } }
I20260809 16:11:27.179415 22437 sys_catalog.cc:458] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:27.179287 22434 sys_catalog.cc:455] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a03ee53cde354ae587586729bce06cff. Latest consensus state: current_term: 1 leader_uuid: "a03ee53cde354ae587586729bce06cff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a03ee53cde354ae587586729bce06cff" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43799 } } }
I20260809 16:11:27.179688 22434 sys_catalog.cc:458] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:27.179855 22451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:27.180497 22451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:27.182026 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.171873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56616 (local address 127.16.224.190:43799)
0809 16:11:27.172070 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:27.172080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:27.172114 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:11:27.172665 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:27.172674 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:27.172740 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:27.172908 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:27.173501 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.174497 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.175239 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.175865 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.175972 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.178346 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:27.178351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:27.178351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:27.178366 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:27.180330 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.180740 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.180743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.180744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.180858 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.181075 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.181076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.181077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.181392 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:27.181533 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:27.181770 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 16:11:27.181886 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260809 16:11:27.182646 22451 catalog_manager.cc:1347] Generated new cluster ID: 9bbf8c8fae1448488def42a3c84db0b9
I20260809 16:11:27.182700 22451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:27.190487 22451 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:27.191004 22451 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:27.194995 22451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a03ee53cde354ae587586729bce06cff: Generated new TSK 0
I20260809 16:11:27.195161 22451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:27.226773 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43799
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:27.311439 22470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:27.311774 22470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:27.311823 22470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:27.313488 22470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:27.313587 22470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:27.313638 22470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:27.315734 22470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:27.316025 22470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:27.316313 22470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:27.322149 22477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:27.322193 22482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:27.322283 22476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:27.322495 22470 server_base.cc:1034] running on GCE node
I20260809 16:11:27.322669 22470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:27.322880 22470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:27.323998 22470 hybrid_clock.cc:648] HybridClock initialized: now 1786291887323988 us; error 26 us; skew 500 ppm
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:27.326367 22470 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:27.326782 22470 webserver.cc:460] Webserver started at http://127.16.224.129:37357/ using document root <none> and password file <none>
I20260809 16:11:27.326982 22470 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:27.327025 22470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:27.327142 22470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:27.328192 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "91b8f1ee802e44df9670b8a6169fd5b3"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "a8d76d4b8a63977742104c62f8b196f6"
server_key_iv: "a49843dedaebea837b67efdb5308a7ff"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.328536 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "91b8f1ee802e44df9670b8a6169fd5b3"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "a8d76d4b8a63977742104c62f8b196f6"
server_key_iv: "a49843dedaebea837b67efdb5308a7ff"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.329802 22470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:27.330499 22494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.330641 22470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:27.330726 22470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "91b8f1ee802e44df9670b8a6169fd5b3"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "a8d76d4b8a63977742104c62f8b196f6"
server_key_iv: "a49843dedaebea837b67efdb5308a7ff"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.330788 22470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:27.341711 22470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:27.343411 22470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:27.343544 22470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:27.343744 22470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:27.344022 22470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:27.344054 22470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.344081 22470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:27.344107 22470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.349723 22470 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:45747
I20260809 16:11:27.349762 22688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:45747 every 8 connection(s)
I20260809 16:11:27.350067 22470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:27.352698 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 22470
I20260809 16:11:27.352787 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:27.352960 17282 external_mini_cluster.cc:1428] Setting key 82fd4761a049bd5d683a6648d29bbcdc
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:27.359586 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.350811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:35975 (local address 127.16.224.190:43799)
0809 16:11:27.350889 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:27.350891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:27.351039 (+ 148us) server_negotiation.cc:407] Connection header received
0809 16:11:27.351543 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:27.351544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:27.351555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:27.351606 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:27.352520 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.353104 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.354259 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.354566 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.354638 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.356906 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:27.356908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:27.356909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:27.356916 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:27.358200 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.358680 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.358681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.358681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.358729 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.358983 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.358983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.358984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.359078 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:27.359121 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:27.359451 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 16:11:27.359521 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:11:27.359836 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.350772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43799 (local address 127.16.224.129:35975)
0809 16:11:27.350921 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:27.350958 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:27.351452 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:27.351663 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:27.351666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:27.352236 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:27.352429 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:27.352434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.353202 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.353203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:27.354171 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:27.354175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.354645 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.354645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:27.354749 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.355242 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:27.355254 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:27.356808 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:27.358281 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:27.358290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:27.358299 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:27.358612 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:27.358788 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:27.358789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:27.358789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:27.358926 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:27.359134 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:27.359140 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:27.359316 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 16:11:27.359436 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 16:11:27.359603 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260809 16:11:27.360363 22689 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43799
I20260809 16:11:27.360464 22689 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:27.360641 22689 heartbeater.cc:511] Master 127.16.224.190:43799 requested a full tablet report, sending...
I20260809 16:11:27.361157 22367 ts_manager.cc:194] Registered new tserver with Master: 91b8f1ee802e44df9670b8a6169fd5b3 (127.16.224.129:45747)
I20260809 16:11:27.361756 22367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:35975
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:27.378043 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43799
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:27.460059 22695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:27.460261 22695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:27.460283 22695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:27.461386 22695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:27.461421 22695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:27.461444 22695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:27.462909 22695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:27.463116 22695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:27.463299 22695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:27.468034 22701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:27.468035 22705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:27.468046 22703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:27.468293 22695 server_base.cc:1034] running on GCE node
I20260809 16:11:27.468492 22695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:27.468727 22695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:27.469856 22695 hybrid_clock.cc:648] HybridClock initialized: now 1786291887469845 us; error 25 us; skew 500 ppm
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:27.472174 22695 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:27.472553 22695 webserver.cc:460] Webserver started at http://127.16.224.130:46257/ using document root <none> and password file <none>
I20260809 16:11:27.472748 22695 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:27.472792 22695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:27.472906 22695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:27.473922 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "4b0fbd4e0a054b0eb7846b53dbe75b34"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "546d459a867d855f97da07df270183a3"
server_key_iv: "3afb6ba87afb4f918d43f0b2f1f316a1"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.474248 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "4b0fbd4e0a054b0eb7846b53dbe75b34"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "546d459a867d855f97da07df270183a3"
server_key_iv: "3afb6ba87afb4f918d43f0b2f1f316a1"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.475513 22695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:27.476218 22720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.476380 22695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:27.476470 22695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "4b0fbd4e0a054b0eb7846b53dbe75b34"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "546d459a867d855f97da07df270183a3"
server_key_iv: "3afb6ba87afb4f918d43f0b2f1f316a1"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.476529 22695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:27.487501 22695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:27.489233 22695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:27.489370 22695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:27.489584 22695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:27.489883 22695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:27.489916 22695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.489944 22695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:27.489962 22695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.495692 22695 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:42603
I20260809 16:11:27.495738 22900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:42603 every 8 connection(s)
I20260809 16:11:27.496052 22695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:27.503347 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 22695
I20260809 16:11:27.503453 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:27.503643 17282 external_mini_cluster.cc:1428] Setting key 7e476fb0ac57af75bdf02df50d2ba989
I20260809 16:11:27.506619 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.496728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:39761 (local address 127.16.224.190:43799)
0809 16:11:27.496791 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:27.496793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:27.497005 (+ 212us) server_negotiation.cc:407] Connection header received
0809 16:11:27.497533 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:27.497534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:27.497547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:27.497596 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:27.498621 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.499118 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.500125 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.500313 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.500369 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.502471 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:27.502473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:27.502473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:27.502480 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:27.503583 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.504034 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.504035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.504035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.504096 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.505930 (+ 1834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.505931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.505931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.506052 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:27.506120 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:27.506442 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 16:11:27.506529 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:11:27.506796 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.496696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43799 (local address 127.16.224.130:39761)
0809 16:11:27.496879 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:27.496918 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:27.497427 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:27.497683 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:27.497687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:27.498273 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:27.498527 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:27.498533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.499206 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.499208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:27.500038 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:27.500041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.500385 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.500386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:27.500507 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.500975 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:27.500983 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:27.502367 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:27.503653 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:27.503661 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:27.503664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:27.503961 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:27.504151 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:27.504152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:27.504152 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:27.504257 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:27.506129 (+ 1872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:27.506137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:27.506275 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 16:11:27.506400 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 16:11:27.506560 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260809 16:11:27.507510 22904 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43799
I20260809 16:11:27.507623 22904 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:27.507813 22904 heartbeater.cc:511] Master 127.16.224.190:43799 requested a full tablet report, sending...
I20260809 16:11:27.508193 22367 ts_manager.cc:194] Registered new tserver with Master: 4b0fbd4e0a054b0eb7846b53dbe75b34 (127.16.224.130:42603)
I20260809 16:11:27.508697 22367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:39761
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:27.527333 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43799
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:27.608691 22912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:27.608887 22912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:27.608909 22912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:27.610051 22912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:27.610095 22912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:27.610121 22912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:27.611580 22912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:43799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:27.611802 22912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:27.612042 22912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:27.616937 22921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:27.616955 22924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:27.616931 22922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:27.617210 22912 server_base.cc:1034] running on GCE node
I20260809 16:11:27.617380 22912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:27.617610 22912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:27.618749 22912 hybrid_clock.cc:648] HybridClock initialized: now 1786291887618735 us; error 28 us; skew 500 ppm
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:27.621343 22912 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:27.621774 22912 webserver.cc:460] Webserver started at http://127.16.224.131:41703/ using document root <none> and password file <none>
I20260809 16:11:27.622000 22912 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:27.622049 22912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:27.622191 22912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:27.623436 22912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "736ec14c014b404d9cd1c691f169dd5d"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "6a07d58e445d397f4d1f6fb03baafd38"
server_key_iv: "68078e367dedab2807757e198990c927"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.623780 22912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "736ec14c014b404d9cd1c691f169dd5d"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "6a07d58e445d397f4d1f6fb03baafd38"
server_key_iv: "68078e367dedab2807757e198990c927"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.625044 22912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:27.625764 22939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.625934 22912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:27.626029 22912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "736ec14c014b404d9cd1c691f169dd5d"
format_stamp: "Formatted at 2026-08-09 16:11:27 on dist-test-slave-fgb3"
server_key: "6a07d58e445d397f4d1f6fb03baafd38"
server_key_iv: "68078e367dedab2807757e198990c927"
server_key_version: "encryptionkey@0"
I20260809 16:11:27.626091 22912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:27.641088 22912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:27.643177 22912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:27.643321 22912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:27.643545 22912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:27.643854 22912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:27.643888 22912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.643918 22912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:27.643940 22912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:27.649595 22912 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:34591
I20260809 16:11:27.649649 23124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:34591 every 8 connection(s)
I20260809 16:11:27.649957 22912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:27.652596 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 22912
I20260809 16:11:27.652697 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:27.652882 17282 external_mini_cluster.cc:1428] Setting key 402dffa46e7713556735459a1180d712
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:27.658746 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.650620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:53497 (local address 127.16.224.190:43799)
0809 16:11:27.650668 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:27.650669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:27.650913 (+ 244us) server_negotiation.cc:407] Connection header received
0809 16:11:27.651428 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:27.651430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:27.651442 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:27.651491 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:27.652422 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.652993 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.654227 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.654497 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.654553 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.656622 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:27.656624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:27.656624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:27.656631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:27.657668 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.658012 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.658013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.658013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.658057 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.658230 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.658231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.658231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.658358 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:27.658412 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:27.658621 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:11:27.658680 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:11:27.658948 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.650631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43799 (local address 127.16.224.131:53497)
0809 16:11:27.650794 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:27.650832 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:27.651330 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:27.651566 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:27.651571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:27.652127 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:27.652325 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:27.652331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.653110 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.653112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:27.654152 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:27.654156 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.654590 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.654591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:27.654707 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.655220 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:27.655229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:27.656545 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:27.657731 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:27.657737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:27.657739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:27.657950 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:27.658101 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:27.658101 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:27.658101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:27.658188 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:27.658403 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:27.658408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:27.658524 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 16:11:27.658606 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:11:27.658723 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260809 16:11:27.659451 23125 heartbeater.cc:347] Connected to a master server at 127.16.224.190:43799
I20260809 16:11:27.659559 23125 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:27.659745 23125 heartbeater.cc:511] Master 127.16.224.190:43799 requested a full tablet report, sending...
I20260809 16:11:27.660149 22367 ts_manager.cc:194] Registered new tserver with Master: 736ec14c014b404d9cd1c691f169dd5d (127.16.224.131:34591)
I20260809 16:11:27.660495 22367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:53497
I20260809 16:11:27.664390 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[22297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291887, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:27.687131 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:27.680612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.16.224.190:43799)
0809 16:11:27.680660 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:27.680661 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:27.680753 (+ 92us) server_negotiation.cc:407] Connection header received
0809 16:11:27.680810 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:27.680811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:27.680821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:27.680883 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:27.681379 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.681862 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.682661 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:27.682838 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:27.682881 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:27.684964 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:27.684966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:27.684966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:27.684974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:27.686223 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.686523 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.686524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.686525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.686575 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:27.686703 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:27.686703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:27.686703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:27.686851 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:27.686922 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:27.686999 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 16:11:27.687063 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:11:27.687382 22367 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56626
I20260809 16:11:27.687809 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 22470
I20260809 16:11:27.707500 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 22695
I20260809 16:11:27.720141 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 22912
I20260809 16:11:27.731810 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 22316
2026-08-09T16:11:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2115 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[23146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[23146](info): set up 2 sockets
Aug 09 16:11:27 dist-test-slave-fgb3 krb5kdc[23146](info): commencing operation
krb5kdc: starting...
W20260809 16:11:28.865221 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:28 dist-test-slave-fgb3 krb5kdc[23146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:28.981148 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33525
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:33525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:29.063599 23170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:29.063799 23170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:29.063819 23170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:29.064888 23170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:29.064919 23170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:29.064931 23170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:29.064942 23170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:29.064952 23170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:29.066469 23170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:33525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:29.066666 23170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:29.066840 23170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:29.071477 23185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:29.071506 23177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:29.071491 23180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:29.071736 23170 server_base.cc:1034] running on GCE node
I20260809 16:11:29.071974 23170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:29.072239 23170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:29.073380 23170 hybrid_clock.cc:648] HybridClock initialized: now 1786291889073366 us; error 29 us; skew 500 ppm
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:29.075806 23170 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:29.076213 23170 webserver.cc:460] Webserver started at http://127.16.224.190:36821/ using document root <none> and password file <none>
I20260809 16:11:29.076431 23170 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:29.076479 23170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:29.076607 23170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:29.077656 23170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "0988ddb805d34bb1be1434daa961f1ba"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "93a32af629ca17e4d2bc1fe098204e1a"
server_key_iv: "fe0e2ba0236ba71fe4105e4b8d400526"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.077986 23170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "0988ddb805d34bb1be1434daa961f1ba"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "93a32af629ca17e4d2bc1fe098204e1a"
server_key_iv: "fe0e2ba0236ba71fe4105e4b8d400526"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.079365 23170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:29.080041 23197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.080226 23170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:29.080343 23170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "0988ddb805d34bb1be1434daa961f1ba"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "93a32af629ca17e4d2bc1fe098204e1a"
server_key_iv: "fe0e2ba0236ba71fe4105e4b8d400526"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.080406 23170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:29.088547 23170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:29.090583 23170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:29.090715 23170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:29.094501 23170 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:33525
I20260809 16:11:29.094542 23280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:33525 every 8 connection(s)
I20260809 16:11:29.095003 23170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:29.095664 23281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:29.096607 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 23170
I20260809 16:11:29.096710 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:29.096865 17282 external_mini_cluster.cc:1428] Setting key b98900dc03e03dcef89635cab20a6430
I20260809 16:11:29.098224 23281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba: Bootstrap starting.
I20260809 16:11:29.098992 23281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:29.099294 23281 log.cc:824] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:29.099922 23281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba: No bootstrap required, opened a new log
I20260809 16:11:29.101545 23281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0988ddb805d34bb1be1434daa961f1ba" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33525 } }
I20260809 16:11:29.101680 23281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:29.101701 23281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0988ddb805d34bb1be1434daa961f1ba, State: Initialized, Role: FOLLOWER
I20260809 16:11:29.101818 23281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0988ddb805d34bb1be1434daa961f1ba" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33525 } }
I20260809 16:11:29.101888 23281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:29.101912 23281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:29.101954 23281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:29.102495 23281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0988ddb805d34bb1be1434daa961f1ba" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33525 } }
I20260809 16:11:29.102595 23281 leader_election.cc:302] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0988ddb805d34bb1be1434daa961f1ba; no voters:
I20260809 16:11:29.102751 23281 leader_election.cc:288] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:29.102797 23290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:29.102931 23290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [term 1 LEADER]: Becoming Leader. State: Replica: 0988ddb805d34bb1be1434daa961f1ba, State: Running, Role: LEADER
I20260809 16:11:29.103012 23281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:29.103029 23290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0988ddb805d34bb1be1434daa961f1ba" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33525 } }
I20260809 16:11:29.103426 23295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0988ddb805d34bb1be1434daa961f1ba. Latest consensus state: current_term: 1 leader_uuid: "0988ddb805d34bb1be1434daa961f1ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0988ddb805d34bb1be1434daa961f1ba" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33525 } } }
I20260809 16:11:29.103526 23295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:29.103731 23299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:29.103986 23294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0988ddb805d34bb1be1434daa961f1ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0988ddb805d34bb1be1434daa961f1ba" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33525 } } }
I20260809 16:11:29.104065 23294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba [sys.catalog]: This master's current role is: LEADER
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:29.105105 23299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:29.106563 23299 catalog_manager.cc:1347] Generated new cluster ID: 0c3bac31b13449c3b5b85e4aafce9080
I20260809 16:11:29.106619 23299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:29.108716 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.097349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.16.224.190:33525)
0809 16:11:29.097507 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:29.097515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:29.097539 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:11:29.098025 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:29.098032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:29.098079 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:29.098191 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:29.099000 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.099944 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.100968 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.102259 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.102377 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.105151 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:29.105155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:29.105155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:29.105167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:29.107152 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.107524 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.107528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.107528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.107634 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.107814 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.107815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.107816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.108120 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:29.108273 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:29.108363 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:11:29.108499 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260809 16:11:29.125190 23299 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:29.125968 23299 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:29.134644 23299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0988ddb805d34bb1be1434daa961f1ba: Generated new TSK 0
I20260809 16:11:29.134902 23299 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:29.161681 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33525
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:29.243604 23326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:29.243817 23326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:29.243839 23326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:29.244988 23326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:29.245033 23326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:29.245059 23326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:29.246986 23326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:29.247232 23326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:29.247469 23326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:29.252300 23341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:29.252314 23337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:29.252319 23336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:29.252528 23326 server_base.cc:1034] running on GCE node
I20260809 16:11:29.252728 23326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:29.252941 23326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:29.254084 23326 hybrid_clock.cc:648] HybridClock initialized: now 1786291889254070 us; error 31 us; skew 500 ppm
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:29.256633 23326 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:29.257068 23326 webserver.cc:460] Webserver started at http://127.16.224.129:44957/ using document root <none> and password file <none>
I20260809 16:11:29.257246 23326 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:29.257284 23326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:29.257376 23326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:29.258462 23326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "de317f9a44504f9184ba9ad2887ee9c8"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "a5eec4a489f103940d0cd0b41dd8e1d0"
server_key_iv: "358a9be375bacf572e34c1536f4be5ff"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.258774 23326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "de317f9a44504f9184ba9ad2887ee9c8"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "a5eec4a489f103940d0cd0b41dd8e1d0"
server_key_iv: "358a9be375bacf572e34c1536f4be5ff"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.259992 23326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:29.260632 23349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.260782 23326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:29.260866 23326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "de317f9a44504f9184ba9ad2887ee9c8"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "a5eec4a489f103940d0cd0b41dd8e1d0"
server_key_iv: "358a9be375bacf572e34c1536f4be5ff"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.260926 23326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:29.269032 23326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:29.271087 23326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:29.271219 23326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:29.271427 23326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:29.271725 23326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:29.271759 23326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.271786 23326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:29.271809 23326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.277387 23326 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:36741
I20260809 16:11:29.277449 23522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:36741 every 8 connection(s)
I20260809 16:11:29.277745 23326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:29.286535 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.278521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:42255 (local address 127.16.224.190:33525)
0809 16:11:29.278575 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:29.278577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:29.278762 (+ 185us) server_negotiation.cc:407] Connection header received
0809 16:11:29.279712 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:29.279713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:29.279726 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:29.279773 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:29.280386 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.280898 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.281877 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.282092 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.282147 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.284181 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:29.284183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:29.284183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:29.284191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:29.285262 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.285628 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.285629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.285630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.285682 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.285894 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.285895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.285896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.286013 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:29.286069 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:29.286351 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 16:11:29.286443 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:11:29.286710 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.278483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33525 (local address 127.16.224.129:42255)
0809 16:11:29.278651 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:29.278687 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:29.279607 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:29.279855 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:29.279859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:29.280109 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:29.280297 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:29.280301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.280987 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.280989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:29.281797 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:29.281799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.282163 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.282163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:29.282278 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.282765 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:29.282773 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:29.284092 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:29.285336 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:29.285343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:29.285346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:29.285562 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:29.285737 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:29.285738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:29.285738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:29.285839 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:29.286069 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:29.286077 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:29.286211 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 16:11:29.286298 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:11:29.286500 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260809 16:11:29.286837 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 23326
I20260809 16:11:29.286954 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:29.287163 17282 external_mini_cluster.cc:1428] Setting key 8fc4ee8ea3db29be2726fa9e37f2cbfa
I20260809 16:11:29.287194 23523 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33525
I20260809 16:11:29.287307 23523 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:29.287490 23523 heartbeater.cc:511] Master 127.16.224.190:33525 requested a full tablet report, sending...
I20260809 16:11:29.287997 23221 ts_manager.cc:194] Registered new tserver with Master: de317f9a44504f9184ba9ad2887ee9c8 (127.16.224.129:36741)
I20260809 16:11:29.288601 23221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:42255
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:29.309893 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33525
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:29.399843 23544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:29.400069 23544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:29.400120 23544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:29.401288 23544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:29.401333 23544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:29.401367 23544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:29.402904 23544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:29.403193 23544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:29.403410 23544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:29.408255 23550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:29.408306 23552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:29.408486 23544 server_base.cc:1034] running on GCE node
W20260809 16:11:29.408535 23549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:29.408743 23544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:29.408980 23544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:29.410136 23544 hybrid_clock.cc:648] HybridClock initialized: now 1786291889410123 us; error 27 us; skew 500 ppm
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:29.412652 23544 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:29.413105 23544 webserver.cc:460] Webserver started at http://127.16.224.130:35971/ using document root <none> and password file <none>
I20260809 16:11:29.413303 23544 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:29.413347 23544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:29.413465 23544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:29.414529 23544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "88d779407b8a44df98838d37bf22da8e"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "cc4a8c870366ca0a08710ca6df731715"
server_key_iv: "879d77e6c18c9a5bc01340260e661861"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.414882 23544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "88d779407b8a44df98838d37bf22da8e"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "cc4a8c870366ca0a08710ca6df731715"
server_key_iv: "879d77e6c18c9a5bc01340260e661861"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.416072 23544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:29.416684 23566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.416828 23544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:29.416906 23544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "88d779407b8a44df98838d37bf22da8e"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "cc4a8c870366ca0a08710ca6df731715"
server_key_iv: "879d77e6c18c9a5bc01340260e661861"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.416967 23544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:29.435153 23544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:29.437073 23544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:29.437216 23544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:29.437454 23544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:29.437791 23544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:29.437829 23544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.437858 23544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:29.437876 23544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.443436 23544 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:39517
I20260809 16:11:29.443487 23756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:39517 every 8 connection(s)
I20260809 16:11:29.443811 23544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:29.445029 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 23544
I20260809 16:11:29.445127 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:29.445303 17282 external_mini_cluster.cc:1428] Setting key e660a6ad294ce020225b268cf5593d3f
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:29.453915 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.444623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:52567 (local address 127.16.224.190:33525)
0809 16:11:29.444680 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:29.444682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:29.444886 (+ 204us) server_negotiation.cc:407] Connection header received
0809 16:11:29.446036 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:29.446037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:29.446049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:29.446110 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:29.446762 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.447424 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.448384 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.448578 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.448635 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.450998 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:29.451000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:29.451000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:29.451009 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:29.452511 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.452945 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.452946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.452946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.453008 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.453276 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.453277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.453278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.453418 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:29.453493 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:29.453735 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 16:11:29.453817 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:11:29.454058 23764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.444579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33525 (local address 127.16.224.130:52567)
0809 16:11:29.444752 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:29.444790 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:29.445836 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:29.446172 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:29.446175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:29.446475 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:29.446681 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:29.446686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.447499 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.447499 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:29.448295 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:29.448298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.448652 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.448652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:29.448797 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.449306 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:29.449315 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:29.450904 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:29.452601 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:29.452610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:29.452612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:29.452868 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:29.453093 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:29.453094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:29.453095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:29.453209 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:29.453491 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:29.453494 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:29.453614 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 16:11:29.453689 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:11:29.453824 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260809 16:11:29.454700 23757 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33525
I20260809 16:11:29.454809 23757 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
I20260809 16:11:29.454990 23757 heartbeater.cc:511] Master 127.16.224.190:33525 requested a full tablet report, sending...
I20260809 16:11:29.455351 23221 ts_manager.cc:194] Registered new tserver with Master: 88d779407b8a44df98838d37bf22da8e (127.16.224.130:39517)
I20260809 16:11:29.455768 23221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:52567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:29.470793 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33525
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:29.553894 23769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:29.554109 23769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:29.554132 23769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:29.555408 23769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:29.555452 23769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:29.555480 23769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:29.557056 23769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:29.557294 23769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:29.557526 23769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:29.562300 23783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:29.562434 23778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:29.562295 23780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:29.562542 23769 server_base.cc:1034] running on GCE node
I20260809 16:11:29.562831 23769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:29.563071 23769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:29.564208 23769 hybrid_clock.cc:648] HybridClock initialized: now 1786291889564195 us; error 31 us; skew 500 ppm
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:29.566740 23769 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:29.567198 23769 webserver.cc:460] Webserver started at http://127.16.224.131:42293/ using document root <none> and password file <none>
I20260809 16:11:29.567430 23769 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:29.567478 23769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:29.567610 23769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:29.568665 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "bd2651a928db42c0b25b96a00235c4f3"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "62bced4a4b7d1bd72d0b6008c4b6ba22"
server_key_iv: "7a1a9634b58e2eefa12f4ccf8c0a9d56"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.569013 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "bd2651a928db42c0b25b96a00235c4f3"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "62bced4a4b7d1bd72d0b6008c4b6ba22"
server_key_iv: "7a1a9634b58e2eefa12f4ccf8c0a9d56"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.570349 23769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:11:29.571116 23791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.571272 23769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:29.571353 23769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "bd2651a928db42c0b25b96a00235c4f3"
format_stamp: "Formatted at 2026-08-09 16:11:29 on dist-test-slave-fgb3"
server_key: "62bced4a4b7d1bd72d0b6008c4b6ba22"
server_key_iv: "7a1a9634b58e2eefa12f4ccf8c0a9d56"
server_key_version: "encryptionkey@0"
I20260809 16:11:29.571411 23769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:29.584937 23769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:29.586779 23769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:29.586905 23769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:29.587102 23769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:29.587363 23769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:29.587393 23769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.587414 23769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:29.587427 23769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:29.592897 23769 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:36769
I20260809 16:11:29.592954 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:36769 every 8 connection(s)
I20260809 16:11:29.593243 23769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:29.595852 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 23769
I20260809 16:11:29.595954 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:29.596138 17282 external_mini_cluster.cc:1428] Setting key 4896c760615731fd07214a22ee9c9008
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:29.602097 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.593952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:36245 (local address 127.16.224.190:33525)
0809 16:11:29.594009 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:29.594011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:29.594194 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:11:29.595175 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:29.595177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:29.595191 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:29.595244 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:29.595903 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.596496 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.597483 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.597683 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.597738 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.599814 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:29.599815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:29.599816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:29.599823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:29.600862 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.601220 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.601222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.601222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.601276 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.601497 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.601498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.601499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.601608 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:29.601664 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:29.601920 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 16:11:29.602009 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:11:29.602262 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.593917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33525 (local address 127.16.224.131:36245)
0809 16:11:29.594068 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:29.594108 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:29.595060 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:29.595329 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:29.595334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:29.595590 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:29.595806 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:29.595811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.596574 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.596576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:29.597393 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:29.597395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.597750 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.597751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:29.597872 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.598369 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:29.598377 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:29.599728 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:29.600937 (+ 1209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:29.600943 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:29.600945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:29.601155 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:29.601330 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:29.601331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:29.601331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:29.601434 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:29.601664 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:29.601671 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:29.601805 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 16:11:29.601891 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:11:29.602043 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260809 16:11:29.602795 24002 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33525
I20260809 16:11:29.602883 24002 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:29.603037 24002 heartbeater.cc:511] Master 127.16.224.190:33525 requested a full tablet report, sending...
I20260809 16:11:29.603361 23221 ts_manager.cc:194] Registered new tserver with Master: bd2651a928db42c0b25b96a00235c4f3 (127.16.224.131:36769)
I20260809 16:11:29.603804 23221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:36245
I20260809 16:11:29.607816 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:29.631567 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.624317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39052 (local address 127.16.224.129:36741)
0809 16:11:29.624511 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:29.624516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:29.624526 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:29.624557 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:29.624559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:29.624573 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:29.624636 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:29.625178 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.625880 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.626614 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.626938 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.626985 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.629010 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:29.629020 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:29.629020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:29.629027 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:29.630534 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.630885 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.630887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.630887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.630937 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.631082 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.631082 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.631082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.631216 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:29.631319 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:29.631396 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 16:11:29.631468 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[23146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:29.639670 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:29.632914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40420 (local address 127.16.224.190:33525)
0809 16:11:29.632982 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:29.632984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:29.632993 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:29.633018 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:29.633019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:29.633030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:29.633075 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:29.633505 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.634002 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.634793 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:29.634994 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:29.635047 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:29.637435 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:29.637437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:29.637437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:29.637451 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:29.638838 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.639117 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.639117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.639118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.639161 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:29.639281 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:29.639281 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:29.639281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:29.639407 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:29.639485 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:29.639546 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 16:11:29.639607 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 16:11:29.639899 23221 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40420
I20260809 16:11:29.640215 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 23326
I20260809 16:11:29.652292 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 23544
I20260809 16:11:29.673012 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 23769
I20260809 16:11:29.686851 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 23170
2026-08-09T16:11:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1955 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[24027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[24027](info): set up 2 sockets
Aug 09 16:11:29 dist-test-slave-fgb3 krb5kdc[24027](info): commencing operation
krb5kdc: starting...
W20260809 16:11:30.827171 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:30 dist-test-slave-fgb3 krb5kdc[24027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:30.943040 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33365
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:33365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:31.023640 24050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:31.023840 24050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:31.023861 24050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:31.024945 24050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:31.024976 24050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:31.024988 24050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:31.024999 24050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:31.025009 24050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:31.026508 24050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:33365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:31.026712 24050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:31.026893 24050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:31.031720 24058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.031728 24059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:31.031795 24050 server_base.cc:1034] running on GCE node
W20260809 16:11:31.031734 24063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:31.032166 24050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:31.032428 24050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:31.033584 24050 hybrid_clock.cc:648] HybridClock initialized: now 1786291891033570 us; error 30 us; skew 500 ppm
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291891, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:31.038793 24050 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:31.039144 24050 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:31.039199 24050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:31.039326 24050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:31.040405 24050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "756c1ca5afa3408f96216740ce8dee47"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "768d5673cf023c2c8a5a105a9129e463"
server_key_iv: "a92efe43cd493c984ad23dd2804a862a"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.040740 24050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "756c1ca5afa3408f96216740ce8dee47"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "768d5673cf023c2c8a5a105a9129e463"
server_key_iv: "a92efe43cd493c984ad23dd2804a862a"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.041980 24050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:11:31.042714 24075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.042872 24050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.042959 24050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "756c1ca5afa3408f96216740ce8dee47"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "768d5673cf023c2c8a5a105a9129e463"
server_key_iv: "a92efe43cd493c984ad23dd2804a862a"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.043018 24050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:31.058347 24050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:31.060322 24050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:31.060464 24050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:31.064353 24050 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:33365
I20260809 16:11:31.064378 24166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:33365 every 8 connection(s)
I20260809 16:11:31.064659 24050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:31.065724 24167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:31.067875 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 24050
I20260809 16:11:31.067965 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:31.068125 17282 external_mini_cluster.cc:1428] Setting key 5ca77c59e5281606a0703a70bb03ce49
I20260809 16:11:31.068135 24167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47: Bootstrap starting.
I20260809 16:11:31.068853 24167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:31.069195 24167 log.cc:824] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:31.069902 24167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47: No bootstrap required, opened a new log
I20260809 16:11:31.072167 24167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756c1ca5afa3408f96216740ce8dee47" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33365 } }
I20260809 16:11:31.072306 24167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:31.072340 24167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756c1ca5afa3408f96216740ce8dee47, State: Initialized, Role: FOLLOWER
I20260809 16:11:31.072474 24167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756c1ca5afa3408f96216740ce8dee47" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33365 } }
I20260809 16:11:31.072548 24167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:31.072583 24167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:31.072628 24167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:31.073271 24167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756c1ca5afa3408f96216740ce8dee47" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33365 } }
I20260809 16:11:31.073372 24167 leader_election.cc:302] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 756c1ca5afa3408f96216740ce8dee47; no voters:
I20260809 16:11:31.073544 24167 leader_election.cc:288] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:31.073592 24173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:31.073729 24173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [term 1 LEADER]: Becoming Leader. State: Replica: 756c1ca5afa3408f96216740ce8dee47, State: Running, Role: LEADER
I20260809 16:11:31.073807 24167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:31.073822 24173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756c1ca5afa3408f96216740ce8dee47" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33365 } }
I20260809 16:11:31.074220 24178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "756c1ca5afa3408f96216740ce8dee47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756c1ca5afa3408f96216740ce8dee47" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33365 } } }
I20260809 16:11:31.074329 24178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:31.074558 24186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:31.075054 24186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:31.075181 24174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 756c1ca5afa3408f96216740ce8dee47. Latest consensus state: current_term: 1 leader_uuid: "756c1ca5afa3408f96216740ce8dee47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756c1ca5afa3408f96216740ce8dee47" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33365 } } }
I20260809 16:11:31.075266 24174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:31.076957 24186 catalog_manager.cc:1347] Generated new cluster ID: c2d6739ce1f1471bbcd32a56376489aa
I20260809 16:11:31.077013 24186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:31.079154 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.068620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.16.224.190:33365)
0809 16:11:31.068822 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.068832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.068867 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:11:31.069419 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.069429 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.069487 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.069605 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:31.070216 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.071605 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.072331 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.072921 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.073051 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.075536 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:31.075541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:31.075541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:31.075553 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:31.077571 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.077931 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.077933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.077934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.078051 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.078227 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.078228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.078228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.078563 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:31.078717 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.078802 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.078937 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260809 16:11:31.083078 24186 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:31.083549 24186 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:31.086715 24186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47: Generated new TSK 0
I20260809 16:11:31.086870 24186 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:31.117225 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33365
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:31.199440 24206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:31.199659 24206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:31.199681 24206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:31.200830 24206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:31.200873 24206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:31.200899 24206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:31.202471 24206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:31.202698 24206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:31.202913 24206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:31.207695 24215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.207737 24212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:31.207955 24206 server_base.cc:1034] running on GCE node
W20260809 16:11:31.207741 24213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:31.208209 24206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:31.208474 24206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:31.209613 24206 hybrid_clock.cc:648] HybridClock initialized: now 1786291891209602 us; error 24 us; skew 500 ppm
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291891, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:31.212162 24206 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:31.212512 24206 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:31.212567 24206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:31.212707 24206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:31.213768 24206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "8f80cd0ab5c94c06957fa0ee6b202645"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "057a253cf2ce462c64d25183a6c6ae7c"
server_key_iv: "aba3e959dd388ac44ea32b8c3d752295"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.214114 24206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "8f80cd0ab5c94c06957fa0ee6b202645"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "057a253cf2ce462c64d25183a6c6ae7c"
server_key_iv: "aba3e959dd388ac44ea32b8c3d752295"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.215461 24206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:31.216112 24226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.216269 24206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:31.216356 24206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "8f80cd0ab5c94c06957fa0ee6b202645"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "057a253cf2ce462c64d25183a6c6ae7c"
server_key_iv: "aba3e959dd388ac44ea32b8c3d752295"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.216426 24206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:31.244846 24206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:31.247135 24206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:31.247279 24206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:31.247524 24206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:31.247833 24206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:31.247869 24206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.247896 24206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:31.247920 24206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.253549 24206 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:43449
I20260809 16:11:31.253600 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:43449 every 8 connection(s)
I20260809 16:11:31.253863 24206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291891, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:31.262463 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.254567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:47323 (local address 127.16.224.190:33365)
0809 16:11:31.254631 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.254633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.254838 (+ 205us) server_negotiation.cc:407] Connection header received
0809 16:11:31.255316 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.255317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.255327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.255376 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:31.256353 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.256865 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.257804 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.258045 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.258108 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.260085 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:31.260087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:31.260087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:31.260094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:31.261274 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.261616 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.261616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.261617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.261671 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.261845 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.261845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.261846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.261945 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:31.261990 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.262249 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.262396 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:11:31.262507 24427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.254552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33365 (local address 127.16.224.129:47323)
0809 16:11:31.254720 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.254759 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.255240 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.255434 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.255438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.255686 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:31.256278 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.256285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.256927 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.256927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.257723 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.257725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.258119 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.258119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.258222 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.258675 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:31.258683 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:31.260004 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:31.261347 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:31.261354 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:31.261356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:31.261559 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:31.261719 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:31.261720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:31.261720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:31.261804 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:31.261986 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:31.261989 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:31.262103 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.262181 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.262296 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260809 16:11:31.262787 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 24206
I20260809 16:11:31.262882 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:31.262990 24425 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33365
I20260809 16:11:31.263072 17282 external_mini_cluster.cc:1428] Setting key 2f500f16d8e46c064ef87ba98cec8456
I20260809 16:11:31.263085 24425 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:31.263231 24425 heartbeater.cc:511] Master 127.16.224.190:33365 requested a full tablet report, sending...
I20260809 16:11:31.263797 24098 ts_manager.cc:194] Registered new tserver with Master: 8f80cd0ab5c94c06957fa0ee6b202645 (127.16.224.129:43449)
I20260809 16:11:31.264407 24098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:47323
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:31.288611 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33365
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:31.374010 24433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:31.374284 24433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:31.374342 24433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:31.375543 24433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:31.375586 24433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:31.375622 24433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:31.377117 24433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:31.377352 24433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:31.377559 24433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:31.382411 24445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.382411 24451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.382476 24446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:31.382529 24433 server_base.cc:1034] running on GCE node
I20260809 16:11:31.382804 24433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:31.383057 24433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:31.384193 24433 hybrid_clock.cc:648] HybridClock initialized: now 1786291891384180 us; error 29 us; skew 500 ppm
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291891, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:31.386549 24433 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:31.386890 24433 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:31.386940 24433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:31.387036 24433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:31.388078 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "f6892f3298d24f9293924b4e0602164a"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "558dc93a8b3ebfb9fa8c997dc258eb71"
server_key_iv: "71711d7c13768fee52fa4eed37920952"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.388388 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "f6892f3298d24f9293924b4e0602164a"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "558dc93a8b3ebfb9fa8c997dc258eb71"
server_key_iv: "71711d7c13768fee52fa4eed37920952"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.389624 24433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:31.390295 24457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.390487 24433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.390574 24433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "f6892f3298d24f9293924b4e0602164a"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "558dc93a8b3ebfb9fa8c997dc258eb71"
server_key_iv: "71711d7c13768fee52fa4eed37920952"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.390630 24433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:31.405078 24433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:31.407220 24433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:31.407337 24433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:31.407541 24433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:31.407804 24433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:31.407833 24433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.407853 24433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:31.407869 24433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.413229 24433 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:39507
I20260809 16:11:31.413296 24643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:39507 every 8 connection(s)
I20260809 16:11:31.413529 24433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:31.414476 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 24433
I20260809 16:11:31.414589 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:31.414773 17282 external_mini_cluster.cc:1428] Setting key 7fa7e310a1149593d0a6b357e872c15b
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291891, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:31.423231 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.414198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:46091 (local address 127.16.224.190:33365)
0809 16:11:31.414258 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.414260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.414514 (+ 254us) server_negotiation.cc:407] Connection header received
0809 16:11:31.415133 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.415134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.415143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.415177 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:31.416340 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.416816 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.417995 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.418201 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.418257 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.420531 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:31.420533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:31.420534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:31.420543 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:31.421921 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.422272 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.422273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.422274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.422364 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.422616 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.422617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.422618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.422759 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:31.422824 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.423060 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.423135 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:11:31.423310 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.414154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33365 (local address 127.16.224.130:46091)
0809 16:11:31.414360 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.414408 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.415041 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.415264 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.415268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.415596 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:31.416231 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.416238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.416900 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.416902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.417911 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.417913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.418272 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.418273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.418432 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.418885 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:31.418895 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:31.420446 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:31.421997 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:31.422003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:31.422004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:31.422204 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:31.422448 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:31.422449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:31.422449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:31.422556 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:31.422826 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:31.422829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:31.422940 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.423018 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.423140 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260809 16:11:31.423794 24644 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33365
I20260809 16:11:31.423904 24644 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:31.424052 24644 heartbeater.cc:511] Master 127.16.224.190:33365 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
I20260809 16:11:31.424466 24098 ts_manager.cc:194] Registered new tserver with Master: f6892f3298d24f9293924b4e0602164a (127.16.224.130:39507)
I20260809 16:11:31.424883 24098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:46091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:31.439173 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33365
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:31.521631 24653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:31.521857 24653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:31.521890 24653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:31.523116 24653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:31.523160 24653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:31.523195 24653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:31.524736 24653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:31.524992 24653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:31.525218 24653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:31.530015 24662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.530002 24663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.530002 24666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:31.530280 24653 server_base.cc:1034] running on GCE node
I20260809 16:11:31.530525 24653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:31.530761 24653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:31.531898 24653 hybrid_clock.cc:648] HybridClock initialized: now 1786291891531885 us; error 24 us; skew 500 ppm
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291891, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:31.534442 24653 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:31.534765 24653 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:31.534813 24653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:31.534940 24653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:31.536018 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "5d38bc04fb434c229c36f96f1accd77b"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "1672b3d2e7d4c13d5f34a5231933e08f"
server_key_iv: "0ccc09ea423247b49678cf1b278914a6"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.536361 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "5d38bc04fb434c229c36f96f1accd77b"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "1672b3d2e7d4c13d5f34a5231933e08f"
server_key_iv: "0ccc09ea423247b49678cf1b278914a6"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.537573 24653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:31.538288 24679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.538465 24653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:31.538553 24653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "5d38bc04fb434c229c36f96f1accd77b"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "1672b3d2e7d4c13d5f34a5231933e08f"
server_key_iv: "0ccc09ea423247b49678cf1b278914a6"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.538625 24653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:31.547610 24653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:31.549353 24653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:31.549489 24653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:31.549707 24653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:31.550019 24653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:31.550052 24653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.550081 24653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:31.550097 24653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.555678 24653 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:46223
I20260809 16:11:31.555732 24882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:46223 every 8 connection(s)
I20260809 16:11:31.556001 24653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:11:31 dist-test-slave-fgb3 krb5kdc[24027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291891, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:31.564317 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.556673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:35337 (local address 127.16.224.190:33365)
0809 16:11:31.556725 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.556727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.556918 (+ 191us) server_negotiation.cc:407] Connection header received
0809 16:11:31.557409 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.557410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.557418 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.557503 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:31.558481 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.558978 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.559937 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.560119 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.560161 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.562183 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:31.562184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:31.562184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:31.562191 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:31.563290 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.563629 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.563630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.563630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.563672 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.563842 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.563842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.563842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.563938 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:31.563982 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.564190 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.564249 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:11:31.564534 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.556638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33365 (local address 127.16.224.131:35337)
0809 16:11:31.556799 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.556840 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.557314 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.557562 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.557565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.557824 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:31.558401 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.558409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.559044 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.559044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.559875 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.559878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.560174 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.560175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.560273 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.560770 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:31.560779 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:31.562104 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:31.563365 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:31.563370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:31.563372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:31.563575 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:31.563714 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:31.563714 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:31.563715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:31.563802 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:31.563979 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:31.563982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:31.564097 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.564172 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.564288 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260809 16:11:31.564648 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 24653
I20260809 16:11:31.564751 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:31.564970 17282 external_mini_cluster.cc:1428] Setting key 3c5899f8cdfeeb17751e8f093319caa5
I20260809 16:11:31.565099 24883 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33365
I20260809 16:11:31.565191 24883 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:31.565337 24883 heartbeater.cc:511] Master 127.16.224.190:33365 requested a full tablet report, sending...
I20260809 16:11:31.565722 24099 ts_manager.cc:194] Registered new tserver with Master: 5d38bc04fb434c229c36f96f1accd77b (127.16.224.131:46223)
I20260809 16:11:31.566090 24099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:35337
I20260809 16:11:31.576547 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:11:31.582895 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.577815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51506 (local address 127.16.224.190:33365)
0809 16:11:31.577856 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.577858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.577946 (+ 88us) server_negotiation.cc:407] Connection header received
0809 16:11:31.578009 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.578010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.578018 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.578084 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:31.578666 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.579148 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.579803 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.579976 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.580015 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.580998 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:31.580999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:31.580999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:31.581005 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:31.582033 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.582305 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.582306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.582306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.582383 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:31.582508 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:31.582509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:31.582509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:31.582636 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:31.582703 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.582768 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.582831 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:11:31.584192 24099 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:31.585678 24099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:31.591558 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.588104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:43449 (local address 127.0.0.1:45900)
0809 16:11:31.588404 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.588413 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.588463 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.588704 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.588706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.588707 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:31.588759 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.588761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.589629 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.589631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.590565 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.590567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.591255 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.591256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.591347 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.591362 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.591426 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.591484 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":52,"threads_started":1}
I20260809 16:11:31.591812 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.588245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.16.224.129:43449)
0809 16:11:31.588432 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.588436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.588446 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:31.588530 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.588532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.588546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.588615 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:31.588839 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.589537 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.590652 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.591178 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.591246 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.591307 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.591686 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.591731 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260809 16:11:31.592545 24900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.588013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45338 (local address 127.16.224.130:39507)
0809 16:11:31.588190 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.588195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.588446 (+ 251us) server_negotiation.cc:407] Connection header received
0809 16:11:31.588569 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.588572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.588586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.588635 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:31.588898 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.589979 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.591441 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.592099 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.592170 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.592224 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.592413 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.592464 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260809 16:11:31.592557 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.587914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:39507 (local address 127.0.0.1:45338)
0809 16:11:31.588369 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.588380 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.588474 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.588760 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.588761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.588764 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:31.588809 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.588811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.590074 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.590074 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.591334 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.591337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.592258 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.592260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.592333 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.592337 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.592389 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.592433 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":76,"thread_start_us":62,"threads_started":1}
I20260809 16:11:31.592558 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.587914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:46223 (local address 127.0.0.1:47416)
0809 16:11:31.588214 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.588256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.588322 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.588648 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.588652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.588654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:31.588723 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.588725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.590072 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.590073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.591334 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.591337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.592293 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.592294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.592356 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.592361 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.592407 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.592450 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":68,"thread_start_us":61,"threads_started":1}
I20260809 16:11:31.592705 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.588207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.16.224.131:46223)
0809 16:11:31.588381 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.588385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.588398 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:11:31.588436 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.588439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.588455 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.588547 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:31.588808 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.589961 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.591418 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.592214 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.592282 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.592334 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.592601 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.592644 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260809 16:11:31.592593 24313 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9dc3d3337741c6b362fa208a9f3650 (DEFAULT_TABLE table=test-table [id=8369e4d6d8894d1392da5721c0e50ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:31.593009 24313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9dc3d3337741c6b362fa208a9f3650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:31.594981 24766 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9dc3d3337741c6b362fa208a9f3650 (DEFAULT_TABLE table=test-table [id=8369e4d6d8894d1392da5721c0e50ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:31.595148 24535 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9dc3d3337741c6b362fa208a9f3650 (DEFAULT_TABLE table=test-table [id=8369e4d6d8894d1392da5721c0e50ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:31.595290 24766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9dc3d3337741c6b362fa208a9f3650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:31.595501 24535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9dc3d3337741c6b362fa208a9f3650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:31.596115 24905 tablet_bootstrap.cc:492] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645: Bootstrap starting.
I20260809 16:11:31.596778 24905 tablet_bootstrap.cc:654] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:31.597458 24905 log.cc:824] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:31.598284 24905 tablet_bootstrap.cc:492] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645: No bootstrap required, opened a new log
I20260809 16:11:31.598387 24905 ts_tablet_manager.cc:1397] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:31.598654 24909 tablet_bootstrap.cc:492] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Bootstrap starting.
I20260809 16:11:31.599422 24909 tablet_bootstrap.cc:654] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:31.599884 24909 log.cc:824] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:31.599952 24905 raft_consensus.cc:348] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.600102 24905 raft_consensus.cc:374] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:31.600137 24905 raft_consensus.cc:729] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f80cd0ab5c94c06957fa0ee6b202645, State: Initialized, Role: FOLLOWER
I20260809 16:11:31.600250 24905 consensus_queue.cc:260] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.600509 24905 ts_tablet_manager.cc:1428] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:11:31.600545 24425 heartbeater.cc:503] Master 127.16.224.190:33365 was elected leader, sending a full tablet report...
I20260809 16:11:31.600602 24909 tablet_bootstrap.cc:492] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: No bootstrap required, opened a new log
I20260809 16:11:31.600648 24909 ts_tablet_manager.cc:1397] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:31.600695 24907 tablet_bootstrap.cc:492] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Bootstrap starting.
I20260809 16:11:31.601269 24907 tablet_bootstrap.cc:654] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:31.601800 24907 log.cc:824] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:31.602553 24907 tablet_bootstrap.cc:492] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: No bootstrap required, opened a new log
I20260809 16:11:31.602615 24907 ts_tablet_manager.cc:1397] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:31.602615 24909 raft_consensus.cc:348] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.602777 24909 raft_consensus.cc:374] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:31.602811 24909 raft_consensus.cc:729] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6892f3298d24f9293924b4e0602164a, State: Initialized, Role: FOLLOWER
I20260809 16:11:31.602922 24909 consensus_queue.cc:260] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.603173 24909 ts_tablet_manager.cc:1428] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:11:31.603191 24644 heartbeater.cc:503] Master 127.16.224.190:33365 was elected leader, sending a full tablet report...
I20260809 16:11:31.604342 24907 raft_consensus.cc:348] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.604480 24907 raft_consensus.cc:374] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:31.604519 24907 raft_consensus.cc:729] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d38bc04fb434c229c36f96f1accd77b, State: Initialized, Role: FOLLOWER
I20260809 16:11:31.604630 24907 consensus_queue.cc:260] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.604868 24907 ts_tablet_manager.cc:1428] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:11:31.604883 24883 heartbeater.cc:503] Master 127.16.224.190:33365 was elected leader, sending a full tablet report...
I20260809 16:11:31.653342 24916 raft_consensus.cc:482] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:31.653474 24916 raft_consensus.cc:504] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.653841 24916 leader_election.cc:288] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d38bc04fb434c229c36f96f1accd77b (127.16.224.131:46223), 8f80cd0ab5c94c06957fa0ee6b202645 (127.16.224.129:43449)
I20260809 16:11:31.656698 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.654098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:58753 (local address 127.16.224.131:46223)
0809 16:11:31.654149 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.654151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.654335 (+ 184us) server_negotiation.cc:407] Connection header received
0809 16:11:31.654375 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.654376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.654386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.654431 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:31.654573 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.655051 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.656014 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.656386 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.656447 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.656456 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.656600 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.656634 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:11:31.656697 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.654096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:34829 (local address 127.16.224.129:43449)
0809 16:11:31.654148 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.654150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.654161 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:31.654247 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.654248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.654258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.654333 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:31.654606 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.655061 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.656023 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.656386 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.656447 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.656456 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.656591 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.656627 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:11:31.656710 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.654037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:43449 (local address 127.16.224.130:34829)
0809 16:11:31.654099 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.654110 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.654157 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.654475 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.654475 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.654477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:31.654531 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.654533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.655140 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.655141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.655934 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.655936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.656458 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.656459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.656530 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.656533 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.656576 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.656624 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":35}
I20260809 16:11:31.656726 24919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.654039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:46223 (local address 127.16.224.130:58753)
0809 16:11:31.654235 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.654245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:31.654309 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:31.654436 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:31.654437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:31.654438 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:31.654504 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.654506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.655123 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.655124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.655933 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:31.655935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.656463 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.656464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:31.656542 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.656545 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:31.656589 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:31.656631 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":14,"thread_start_us":96,"threads_started":1}
I20260809 16:11:31.657102 24347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dc3d3337741c6b362fa208a9f3650" candidate_uuid: "f6892f3298d24f9293924b4e0602164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" is_pre_election: true
I20260809 16:11:31.657182 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dc3d3337741c6b362fa208a9f3650" candidate_uuid: "f6892f3298d24f9293924b4e0602164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d38bc04fb434c229c36f96f1accd77b" is_pre_election: true
I20260809 16:11:31.657279 24347 raft_consensus.cc:2454] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6892f3298d24f9293924b4e0602164a in term 0.
I20260809 16:11:31.657301 24796 raft_consensus.cc:2454] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6892f3298d24f9293924b4e0602164a in term 0.
I20260809 16:11:31.657505 24462 leader_election.cc:302] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d38bc04fb434c229c36f96f1accd77b, f6892f3298d24f9293924b4e0602164a; no voters:
I20260809 16:11:31.657634 24916 raft_consensus.cc:2781] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:31.657702 24916 raft_consensus.cc:482] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:31.657719 24916 raft_consensus.cc:3037] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:31.658468 24916 raft_consensus.cc:504] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.658591 24916 leader_election.cc:288] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [CANDIDATE]: Term 1 election: Requested vote from peers 5d38bc04fb434c229c36f96f1accd77b (127.16.224.131:46223), 8f80cd0ab5c94c06957fa0ee6b202645 (127.16.224.129:43449)
I20260809 16:11:31.658735 24347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dc3d3337741c6b362fa208a9f3650" candidate_uuid: "f6892f3298d24f9293924b4e0602164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f80cd0ab5c94c06957fa0ee6b202645"
I20260809 16:11:31.658747 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9dc3d3337741c6b362fa208a9f3650" candidate_uuid: "f6892f3298d24f9293924b4e0602164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d38bc04fb434c229c36f96f1accd77b"
I20260809 16:11:31.658815 24796 raft_consensus.cc:3037] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:31.658814 24347 raft_consensus.cc:3037] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:31.659672 24347 raft_consensus.cc:2454] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6892f3298d24f9293924b4e0602164a in term 1.
I20260809 16:11:31.659672 24796 raft_consensus.cc:2454] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6892f3298d24f9293924b4e0602164a in term 1.
I20260809 16:11:31.659835 24461 leader_election.cc:302] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f80cd0ab5c94c06957fa0ee6b202645, f6892f3298d24f9293924b4e0602164a; no voters:
I20260809 16:11:31.659927 24916 raft_consensus.cc:2781] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:31.660110 24916 raft_consensus.cc:686] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 1 LEADER]: Becoming Leader. State: Replica: f6892f3298d24f9293924b4e0602164a, State: Running, Role: LEADER
I20260809 16:11:31.660215 24916 consensus_queue.cc:237] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } }
I20260809 16:11:31.660988 24099 catalog_manager.cc:5495] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a reported cstate change: term changed from 0 to 1, leader changed from <none> to f6892f3298d24f9293924b4e0602164a (127.16.224.130). New cstate: current_term: 1 leader_uuid: "f6892f3298d24f9293924b4e0602164a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6892f3298d24f9293924b4e0602164a" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 } health_report { overall_health: UNKNOWN } } }
W20260809 16:11:31.663983 24645 tablet.cc:2369] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:31.682744 24900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.680549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45344 (local address 127.16.224.130:39507)
0809 16:11:31.680623 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.680625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.680632 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:31.680670 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.680671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.680691 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.680755 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:31.680881 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.681353 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.682055 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:31.682251 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:31.682301 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:31.682460 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:31.682568 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:31.682628 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.682642 (+ 14us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.682660 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:11:31.684557 24796 raft_consensus.cc:1261] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 1 FOLLOWER]: Refusing update from remote peer f6892f3298d24f9293924b4e0602164a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:31.684566 24347 raft_consensus.cc:1261] T 3c9dc3d3337741c6b362fa208a9f3650 P 8f80cd0ab5c94c06957fa0ee6b202645 [term 1 FOLLOWER]: Refusing update from remote peer f6892f3298d24f9293924b4e0602164a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:31.684893 24920 consensus_queue.cc:1035] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f80cd0ab5c94c06957fa0ee6b202645" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 43449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:31.685039 24916 consensus_queue.cc:1035] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d38bc04fb434c229c36f96f1accd77b" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:31.687471 24925 mvcc.cc:205] Tried to move back new op lower bound from 7316651588337266688 to 7316651588240883712. Current Snapshot: MvccSnapshot[applied={T|T < 7316651588337266688}]
I20260809 16:11:31.687925 24928 mvcc.cc:205] Tried to move back new op lower bound from 7316651588337266688 to 7316651588240883712. Current Snapshot: MvccSnapshot[applied={T|T < 7316651588337266688}]
I20260809 16:11:31.688297 24926 mvcc.cc:205] Tried to move back new op lower bound from 7316651588337266688 to 7316651588240883712. Current Snapshot: MvccSnapshot[applied={T|T < 7316651588337266688}]
I20260809 16:11:31.689828 24098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:31.689931 24098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:31.690706 24098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 756c1ca5afa3408f96216740ce8dee47: Sending DeleteTablet for 3 replicas of tablet 3c9dc3d3337741c6b362fa208a9f3650
I20260809 16:11:31.690989 24766 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9dc3d3337741c6b362fa208a9f3650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:31 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:47416
I20260809 16:11:31.691051 24535 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9dc3d3337741c6b362fa208a9f3650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:31 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:45338
I20260809 16:11:31.691274 24313 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c9dc3d3337741c6b362fa208a9f3650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:31 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:45900
I20260809 16:11:31.691370 24940 tablet_replica.cc:330] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: stopping tablet replica
I20260809 16:11:31.691383 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 24206
I20260809 16:11:31.691468 24940 raft_consensus.cc:2229] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:31.691577 24940 raft_consensus.cc:2258] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:31.691591 24943 tablet_replica.cc:330] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: stopping tablet replica
I20260809 16:11:31.691826 24943 raft_consensus.cc:2229] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:31.691970 24943 raft_consensus.cc:2258] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:31.696111 24943 ts_tablet_manager.cc:1905] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:31.698223 24943 ts_tablet_manager.cc:1918] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:31.698419 24943 log.cc:1196] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/3c9dc3d3337741c6b362fa208a9f3650
I20260809 16:11:31.698884 24943 ts_tablet_manager.cc:1939] T 3c9dc3d3337741c6b362fa208a9f3650 P f6892f3298d24f9293924b4e0602164a: Deleting consensus metadata
I20260809 16:11:31.699518 24079 catalog_manager.cc:4841] TS f6892f3298d24f9293924b4e0602164a (127.16.224.130:39507): tablet 3c9dc3d3337741c6b362fa208a9f3650 (table test-table [id=8369e4d6d8894d1392da5721c0e50ae3]) successfully deleted
I20260809 16:11:31.704434 24940 ts_tablet_manager.cc:1905] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:31.706079 24940 ts_tablet_manager.cc:1918] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:31.706174 24940 log.cc:1196] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/3c9dc3d3337741c6b362fa208a9f3650
I20260809 16:11:31.706475 24940 ts_tablet_manager.cc:1939] T 3c9dc3d3337741c6b362fa208a9f3650 P 5d38bc04fb434c229c36f96f1accd77b: Deleting consensus metadata
I20260809 16:11:31.706969 24080 catalog_manager.cc:4841] TS 5d38bc04fb434c229c36f96f1accd77b (127.16.224.131:46223): tablet 3c9dc3d3337741c6b362fa208a9f3650 (table test-table [id=8369e4d6d8894d1392da5721c0e50ae3]) successfully deleted
W20260809 16:11:31.710990 24079 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:43449 (error 108)
I20260809 16:11:31.711521 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.711379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:43449 (local address 127.0.0.1:45910)
0809 16:11:31.711440 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:31.711490 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:11:31.711549 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 24433
W20260809 16:11:31.719631 24079 catalog_manager.cc:4584] TS 8f80cd0ab5c94c06957fa0ee6b202645 (127.16.224.129:43449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c9dc3d3337741c6b362fa208a9f3650: Network error: Client connection negotiation failed: client connection to 127.16.224.129:43449: connect: Connection refused (error 111)
I20260809 16:11:31.724062 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 24653
I20260809 16:11:31.732668 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 24050
2026-08-09T16:11:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2047 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T16:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:31Z Disabled control of system clock
I20260809 16:11:31.767760 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33411
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:33411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:11:31.849083 24954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:31.849308 24954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:31.849329 24954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:31.850457 24954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:31.850502 24954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:31.850518 24954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:31.850530 24954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:31.850541 24954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:31.851866 24954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:33411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:31.852072 24954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:31.852349 24954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:31.857002 24968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.857023 24964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:31.856976 24966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:31.857190 24954 server_base.cc:1034] running on GCE node
I20260809 16:11:31.857417 24954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:31.857685 24954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:31.858819 24954 hybrid_clock.cc:648] HybridClock initialized: now 1786291891858806 us; error 29 us; skew 500 ppm
I20260809 16:11:31.863869 24954 webserver.cc:460] Webserver started at http://127.16.224.190:38063/ using document root <none> and password file <none>
I20260809 16:11:31.864087 24954 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:31.864132 24954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:31.864286 24954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:31.865362 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "1196246359474011a48016ae6bb5ec84"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "1e6bb1bdb7a84992293f005d3bdc76be"
server_key_iv: "d464c5c3449f9fa4b2aaf9f7fec400b9"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.865694 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "1196246359474011a48016ae6bb5ec84"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "1e6bb1bdb7a84992293f005d3bdc76be"
server_key_iv: "d464c5c3449f9fa4b2aaf9f7fec400b9"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.867036 24954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:31.867734 24979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:31.867897 24954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:31.867985 24954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "1196246359474011a48016ae6bb5ec84"
format_stamp: "Formatted at 2026-08-09 16:11:31 on dist-test-slave-fgb3"
server_key: "1e6bb1bdb7a84992293f005d3bdc76be"
server_key_iv: "d464c5c3449f9fa4b2aaf9f7fec400b9"
server_key_version: "encryptionkey@0"
I20260809 16:11:31.868048 24954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:31.871361 24954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:31.871601 24954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:31.871723 24954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:31.875512 24954 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:33411
I20260809 16:11:31.875554 25058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:33411 every 8 connection(s)
I20260809 16:11:31.875855 24954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:31.876847 25061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:31.879264 25061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84: Bootstrap starting.
I20260809 16:11:31.879824 25061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:31.880092 25061 log.cc:824] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:31.880756 25061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84: No bootstrap required, opened a new log
I20260809 16:11:31.881973 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 24954
I20260809 16:11:31.882066 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:31.882225 17282 external_mini_cluster.cc:1428] Setting key 34419b979d8263b803152a7711f65c94
I20260809 16:11:31.882464 25061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1196246359474011a48016ae6bb5ec84" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33411 } }
I20260809 16:11:31.882627 25061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:31.882663 25061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1196246359474011a48016ae6bb5ec84, State: Initialized, Role: FOLLOWER
I20260809 16:11:31.882804 25061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1196246359474011a48016ae6bb5ec84" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33411 } }
I20260809 16:11:31.882879 25061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:31.882915 25061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:31.882956 25061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:31.883486 25061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1196246359474011a48016ae6bb5ec84" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33411 } }
I20260809 16:11:31.883598 25061 leader_election.cc:302] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1196246359474011a48016ae6bb5ec84; no voters:
I20260809 16:11:31.883752 25061 leader_election.cc:288] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:31.883932 25070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:31.884013 25061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:31.884081 25070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [term 1 LEADER]: Becoming Leader. State: Replica: 1196246359474011a48016ae6bb5ec84, State: Running, Role: LEADER
I20260809 16:11:31.884181 25070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1196246359474011a48016ae6bb5ec84" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33411 } }
I20260809 16:11:31.884624 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:31.882754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.16.224.190:33411)
0809 16:11:31.883015 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:31.883024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:31.883053 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:11:31.883155 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:31.883161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:31.883195 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:31.883309 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:31.884112 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:31.884117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:31.884117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:31.884122 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:31.884173 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:31.884188 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:31.884229 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:31.884355 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 16:11:31.884421 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260809 16:11:31.884634 25071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1196246359474011a48016ae6bb5ec84. Latest consensus state: current_term: 1 leader_uuid: "1196246359474011a48016ae6bb5ec84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1196246359474011a48016ae6bb5ec84" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33411 } } }
I20260809 16:11:31.884742 25071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:31.884975 25081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:31.885628 25081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:31.885632 25069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1196246359474011a48016ae6bb5ec84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1196246359474011a48016ae6bb5ec84" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 33411 } } }
I20260809 16:11:31.885747 25069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:31.887271 25081 catalog_manager.cc:1347] Generated new cluster ID: 8d62411aa4b24e319ae7abc8b9b53ad1
I20260809 16:11:31.887318 25081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:31.905270 25081 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:31.905460 25081 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:31.910656 25081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84: Generated new TSK 0
I20260809 16:11:31.910842 25081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:11:31.921305 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33411
--builtin_ntp_servers=127.16.224.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:11:32.015726 25102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:32.015930 25102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:32.015952 25102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:32.017102 25102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:32.017141 25102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:32.017165 25102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:32.018569 25102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:32.018814 25102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:32.019107 25102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:32.024116 25114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:32.024087 25118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:32.024116 25113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.024257 25102 server_base.cc:1034] running on GCE node
I20260809 16:11:32.024587 25102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:32.024824 25102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:32.025971 25102 hybrid_clock.cc:648] HybridClock initialized: now 1786291892025957 us; error 30 us; skew 500 ppm
I20260809 16:11:32.027205 25102 webserver.cc:460] Webserver started at http://127.16.224.129:43269/ using document root <none> and password file <none>
I20260809 16:11:32.027462 25102 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:32.027508 25102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:32.027623 25102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:32.028663 25102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "bece2fdc034e46ccbf299b80aff10fd8"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "04332ed4805385f6fd6b88fc60ef24a8"
server_key_iv: "502567b1f9c68275395e896e74d48578"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.028978 25102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "bece2fdc034e46ccbf299b80aff10fd8"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "04332ed4805385f6fd6b88fc60ef24a8"
server_key_iv: "502567b1f9c68275395e896e74d48578"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.030200 25102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:32.030962 25126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.031147 25102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:32.031243 25102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "bece2fdc034e46ccbf299b80aff10fd8"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "04332ed4805385f6fd6b88fc60ef24a8"
server_key_iv: "502567b1f9c68275395e896e74d48578"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.031301 25102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:32.034597 25102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:32.034833 25102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:32.034930 25102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:32.035104 25102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:32.035358 25102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:32.035387 25102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.035406 25102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:32.035424 25102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.040840 25102 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:39855
I20260809 16:11:32.040910 25316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:39855 every 8 connection(s)
I20260809 16:11:32.041230 25102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:32.044160 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.041985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:38783 (local address 127.16.224.190:33411)
0809 16:11:32.042041 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.042043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.042270 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:11:32.042818 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.042820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.042827 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.042868 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.043981 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.043982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.043982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.043985 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.043997 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.044000 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.044035 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.044090 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.044102 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:11:32.044355 25319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.041955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33411 (local address 127.16.224.129:38783)
0809 16:11:32.042151 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.042193 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.042707 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.042941 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.042945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.042953 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.043849 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.043850 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.043880 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.043911 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.044041 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.044043 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.044045 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.044094 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.044144 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260809 16:11:32.044914 25317 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33411
I20260809 16:11:32.045004 25317 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:32.045150 25317 heartbeater.cc:511] Master 127.16.224.190:33411 requested a full tablet report, sending...
I20260809 16:11:32.045708 25005 ts_manager.cc:194] Registered new tserver with Master: bece2fdc034e46ccbf299b80aff10fd8 (127.16.224.129:39855)
I20260809 16:11:32.050786 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 25102
I20260809 16:11:32.050869 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:32.051009 17282 external_mini_cluster.cc:1428] Setting key 2e1904feaa79afdcd741a2d64ac50e82
I20260809 16:11:32.051944 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33411
--builtin_ntp_servers=127.16.224.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:11:32.135569 25321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:32.135799 25321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:32.135821 25321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:32.137051 25321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:32.137096 25321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:32.137122 25321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:32.138563 25321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:32.138804 25321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:32.139109 25321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:32.143966 25334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.144017 25321 server_base.cc:1034] running on GCE node
W20260809 16:11:32.143978 25339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:32.143980 25336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.144388 25321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:32.144636 25321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:32.145776 25321 hybrid_clock.cc:648] HybridClock initialized: now 1786291892145762 us; error 31 us; skew 500 ppm
I20260809 16:11:32.147109 25321 webserver.cc:460] Webserver started at http://127.16.224.130:33423/ using document root <none> and password file <none>
I20260809 16:11:32.147328 25321 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:32.147388 25321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:32.147519 25321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:32.148603 25321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "1bdd0b617a74435e9beb09ce0526a186"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "5adf72b2c70c1a027eaa0257b23dcf1a"
server_key_iv: "22b9040f6822a22887aff5f86f2391d1"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.148957 25321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "1bdd0b617a74435e9beb09ce0526a186"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "5adf72b2c70c1a027eaa0257b23dcf1a"
server_key_iv: "22b9040f6822a22887aff5f86f2391d1"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.150205 25321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:32.150978 25347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.151156 25321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:11:32.151247 25321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "1bdd0b617a74435e9beb09ce0526a186"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "5adf72b2c70c1a027eaa0257b23dcf1a"
server_key_iv: "22b9040f6822a22887aff5f86f2391d1"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.151315 25321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:32.154572 25321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:32.154821 25321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:32.154937 25321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:32.155122 25321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:32.155419 25321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:32.155452 25321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.155478 25321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:32.155504 25321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.161114 25321 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:33629
I20260809 16:11:32.161154 25533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:33629 every 8 connection(s)
I20260809 16:11:32.161473 25321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:32.164196 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.162150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:35503 (local address 127.16.224.190:33411)
0809 16:11:32.162218 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.162220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.162433 (+ 213us) server_negotiation.cc:407] Connection header received
0809 16:11:32.162901 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.162902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.162907 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.162964 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.164024 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.164025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.164025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.164028 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.164040 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.164044 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.164082 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.164127 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.164138 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:11:32.164362 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.162108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33411 (local address 127.16.224.130:35503)
0809 16:11:32.162295 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.162362 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.162822 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.163029 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.163032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.163040 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.163903 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.163903 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.163930 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.163958 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.164082 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.164084 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.164087 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.164134 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.164179 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260809 16:11:32.164788 25534 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33411
I20260809 16:11:32.164871 25534 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:32.164983 25534 heartbeater.cc:511] Master 127.16.224.190:33411 requested a full tablet report, sending...
I20260809 16:11:32.165315 25005 ts_manager.cc:194] Registered new tserver with Master: 1bdd0b617a74435e9beb09ce0526a186 (127.16.224.130:33629)
I20260809 16:11:32.167418 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 25321
I20260809 16:11:32.167511 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:32.167675 17282 external_mini_cluster.cc:1428] Setting key 70f55898ed2630285480287d9817e530
I20260809 16:11:32.168541 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33411
--builtin_ntp_servers=127.16.224.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:11:32.250949 25537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:32.251164 25537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:32.251192 25537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:32.252314 25537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:32.252347 25537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:32.252370 25537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:32.253723 25537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:33411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:32.253984 25537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:32.254273 25537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:32.259143 25547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:32.259143 25545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:32.259131 25543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.259382 25537 server_base.cc:1034] running on GCE node
I20260809 16:11:32.259586 25537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:32.259821 25537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:32.260990 25537 hybrid_clock.cc:648] HybridClock initialized: now 1786291892260971 us; error 29 us; skew 500 ppm
I20260809 16:11:32.262274 25537 webserver.cc:460] Webserver started at http://127.16.224.131:45341/ using document root <none> and password file <none>
I20260809 16:11:32.262540 25537 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:32.262593 25537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:32.262718 25537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:32.263767 25537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "eb876e5affa24339b412712675222eab"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "33696bec941f9c04b178340102bf1c8e"
server_key_iv: "f6f70d8b234bce8084df37686e9d933b"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.264109 25537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "eb876e5affa24339b412712675222eab"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "33696bec941f9c04b178340102bf1c8e"
server_key_iv: "f6f70d8b234bce8084df37686e9d933b"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.265362 25537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:32.266003 25558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.266155 25537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:32.266229 25537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "eb876e5affa24339b412712675222eab"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "33696bec941f9c04b178340102bf1c8e"
server_key_iv: "f6f70d8b234bce8084df37686e9d933b"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.266294 25537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:32.269548 25537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:32.269786 25537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:32.269902 25537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:32.270100 25537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:32.270417 25537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:32.270454 25537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.270479 25537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:32.270503 25537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.275813 25537 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:42511
I20260809 16:11:32.275871 25740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:42511 every 8 connection(s)
I20260809 16:11:32.276139 25537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:32.278748 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.276785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:59381 (local address 127.16.224.190:33411)
0809 16:11:32.276861 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.276864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.277048 (+ 184us) server_negotiation.cc:407] Connection header received
0809 16:11:32.277468 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.277469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.277475 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.277525 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.278572 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.278574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.278574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.278578 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.278594 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.278598 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.278642 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.278683 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.278695 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:11:32.278906 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.276760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:33411 (local address 127.16.224.131:59381)
0809 16:11:32.276934 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.276973 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.277393 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.277578 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.277581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.277589 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.278441 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.278442 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.278470 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.278499 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.278640 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.278642 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.278644 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.278690 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.278735 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260809 16:11:32.279361 25742 heartbeater.cc:347] Connected to a master server at 127.16.224.190:33411
I20260809 16:11:32.279435 25742 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:32.279546 25742 heartbeater.cc:511] Master 127.16.224.190:33411 requested a full tablet report, sending...
I20260809 16:11:32.279846 25005 ts_manager.cc:194] Registered new tserver with Master: eb876e5affa24339b412712675222eab (127.16.224.131:42511)
I20260809 16:11:32.284237 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 25537
I20260809 16:11:32.284327 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:32.284521 17282 external_mini_cluster.cc:1428] Setting key 194341c6be35b62e9b521e2b289536a4
I20260809 16:11:32.285310 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:11:32.287322 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.286569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.16.224.190:33411)
0809 16:11:32.286637 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.286639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.286694 (+ 55us) server_negotiation.cc:407] Connection header received
0809 16:11:32.286742 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.286743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.286746 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.286784 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.287171 (+ 387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.287172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.287173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.287176 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.287188 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.287191 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.287227 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.287255 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.287266 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:11:32.288605 25005 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:32.290072 25005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:32.293531 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.292065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:33629 (local address 127.0.0.1:43668)
0809 16:11:32.292408 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.292416 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.292446 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.292634 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.292641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.292646 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.293107 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.293108 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.293123 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.293143 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.293375 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.293383 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.293386 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.293453 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.293468 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":64,"threads_started":1}
I20260809 16:11:32.293740 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.292301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.16.224.130:33629)
0809 16:11:32.292493 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.292497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.292505 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:32.292532 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.292534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.292544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.292626 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.293233 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.293242 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.293243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.293248 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.293290 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.293317 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.293376 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.293469 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.293486 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
I20260809 16:11:32.293751 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.292060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:42511 (local address 127.0.0.1:59368)
0809 16:11:32.292237 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.292276 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.292331 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.293103 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.293104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.293106 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.293419 (+ 313us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.293420 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.293424 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.293432 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.293651 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.293652 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.293652 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.293678 (+ 26us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.293690 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260809 16:11:32.293836 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.292060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:39855 (local address 127.0.0.1:42860)
0809 16:11:32.292432 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.292440 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.292493 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.292635 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.292641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.292646 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.293275 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.293275 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.293281 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.293290 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.293676 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.293677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.293677 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.293723 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.293735 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":17,"thread_start_us":79,"threads_started":1}
I20260809 16:11:32.293833 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.292247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.16.224.129:39855)
0809 16:11:32.292423 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.292427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.292494 (+ 67us) server_negotiation.cc:407] Connection header received
0809 16:11:32.292536 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.292538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.292546 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.292625 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.293536 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.293540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.293540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.293543 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.293576 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.293591 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.293647 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.293737 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.293752 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260809 16:11:32.294039 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.292211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59368 (local address 127.16.224.131:42511)
0809 16:11:32.292971 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.292975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.292984 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:32.293007 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.293009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.293017 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.293089 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.293511 (+ 422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.293518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.293518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.293522 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.293553 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.293576 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.293922 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.293948 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.293963 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":69,"threads_started":1}
I20260809 16:11:32.294584 25639 tablet_service.cc:1467] Processing CreateTablet for tablet c8e2209a35c348ecb0dd22d380c9416f (DEFAULT_TABLE table=test-table [id=98d9fc45e6ca47fe9a58fc595f9f166e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:32.294546 25215 tablet_service.cc:1467] Processing CreateTablet for tablet c8e2209a35c348ecb0dd22d380c9416f (DEFAULT_TABLE table=test-table [id=98d9fc45e6ca47fe9a58fc595f9f166e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:32.294917 25639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e2209a35c348ecb0dd22d380c9416f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:32.294919 25215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e2209a35c348ecb0dd22d380c9416f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:32.298143 25769 tablet_bootstrap.cc:492] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8: Bootstrap starting.
I20260809 16:11:32.298457 25768 tablet_bootstrap.cc:492] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Bootstrap starting.
I20260809 16:11:32.298703 25427 tablet_service.cc:1467] Processing CreateTablet for tablet c8e2209a35c348ecb0dd22d380c9416f (DEFAULT_TABLE table=test-table [id=98d9fc45e6ca47fe9a58fc595f9f166e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:32.299021 25427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e2209a35c348ecb0dd22d380c9416f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:32.299028 25768 tablet_bootstrap.cc:654] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:32.299289 25768 log.cc:824] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:32.300017 25769 tablet_bootstrap.cc:654] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:32.300027 25768 tablet_bootstrap.cc:492] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: No bootstrap required, opened a new log
I20260809 16:11:32.300088 25768 ts_tablet_manager.cc:1397] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:32.300311 25769 log.cc:824] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:32.300956 25769 tablet_bootstrap.cc:492] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8: No bootstrap required, opened a new log
I20260809 16:11:32.301016 25769 ts_tablet_manager.cc:1397] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:11:32.302275 25772 tablet_bootstrap.cc:492] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Bootstrap starting.
I20260809 16:11:32.302592 25768 raft_consensus.cc:348] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.302727 25768 raft_consensus.cc:374] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:32.302747 25768 raft_consensus.cc:729] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb876e5affa24339b412712675222eab, State: Initialized, Role: FOLLOWER
I20260809 16:11:32.302873 25768 consensus_queue.cc:260] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.303117 25768 ts_tablet_manager.cc:1428] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:11:32.303144 25742 heartbeater.cc:503] Master 127.16.224.190:33411 was elected leader, sending a full tablet report...
I20260809 16:11:32.303346 25772 tablet_bootstrap.cc:654] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:32.303406 25769 raft_consensus.cc:348] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.303574 25769 raft_consensus.cc:374] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:32.303617 25769 raft_consensus.cc:729] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bece2fdc034e46ccbf299b80aff10fd8, State: Initialized, Role: FOLLOWER
I20260809 16:11:32.303676 25772 log.cc:824] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:32.303732 25769 consensus_queue.cc:260] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.304101 25769 ts_tablet_manager.cc:1428] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:11:32.304229 25317 heartbeater.cc:503] Master 127.16.224.190:33411 was elected leader, sending a full tablet report...
I20260809 16:11:32.304481 25772 tablet_bootstrap.cc:492] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: No bootstrap required, opened a new log
I20260809 16:11:32.304551 25772 ts_tablet_manager.cc:1397] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:11:32.306501 25772 raft_consensus.cc:348] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.306619 25772 raft_consensus.cc:374] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:32.306639 25772 raft_consensus.cc:729] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bdd0b617a74435e9beb09ce0526a186, State: Initialized, Role: FOLLOWER
I20260809 16:11:32.306743 25772 consensus_queue.cc:260] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.306959 25772 ts_tablet_manager.cc:1428] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 16:11:32.307008 25534 heartbeater.cc:503] Master 127.16.224.190:33411 was elected leader, sending a full tablet report...
W20260809 16:11:32.411993 25535 tablet.cc:2369] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:32.421350 25773 raft_consensus.cc:482] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:32.421475 25773 raft_consensus.cc:504] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.421855 25773 leader_election.cc:288] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bece2fdc034e46ccbf299b80aff10fd8 (127.16.224.129:39855), 1bdd0b617a74435e9beb09ce0526a186 (127.16.224.130:33629)
I20260809 16:11:32.423115 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.422039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:33629 (local address 127.16.224.131:40397)
0809 16:11:32.422095 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.422108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.422165 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.422356 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.422357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.422360 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.422780 (+ 420us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.422780 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.422789 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.422804 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.422990 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.422990 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.422991 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.423034 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.423048 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:11:32.423122 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.422110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:40397 (local address 127.16.224.130:33629)
0809 16:11:32.422203 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.422205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.422215 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:32.422248 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.422249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.422256 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.422347 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.422895 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.422897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.422897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.422902 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.422921 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.422926 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.422987 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.423044 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.423058 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 16:11:32.423251 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.422198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:42911 (local address 127.16.224.129:39855)
0809 16:11:32.422249 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.422251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.422332 (+ 81us) server_negotiation.cc:407] Connection header received
0809 16:11:32.422419 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.422420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.422426 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.422467 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.423042 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.423043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.423044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.423048 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.423067 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.423072 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.423125 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.423173 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.423185 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:11:32.423264 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.422088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:39855 (local address 127.16.224.131:42911)
0809 16:11:32.422273 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.422281 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.422357 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.422477 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.422478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.422480 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:32.422947 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:11:32.422948 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:32.422954 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:11:32.422966 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:32.423127 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:32.423128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:32.423129 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:11:32.423173 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:32.423185 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260809 16:11:32.423616 25457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e2209a35c348ecb0dd22d380c9416f" candidate_uuid: "eb876e5affa24339b412712675222eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bdd0b617a74435e9beb09ce0526a186" is_pre_election: true
I20260809 16:11:32.423739 25457 raft_consensus.cc:2454] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb876e5affa24339b412712675222eab in term 0.
I20260809 16:11:32.423974 25565 leader_election.cc:302] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bdd0b617a74435e9beb09ce0526a186, eb876e5affa24339b412712675222eab; no voters:
I20260809 16:11:32.424029 25243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e2209a35c348ecb0dd22d380c9416f" candidate_uuid: "eb876e5affa24339b412712675222eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bece2fdc034e46ccbf299b80aff10fd8" is_pre_election: true
I20260809 16:11:32.424120 25773 raft_consensus.cc:2781] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:32.424171 25773 raft_consensus.cc:482] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:32.424181 25243 raft_consensus.cc:2454] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb876e5affa24339b412712675222eab in term 0.
I20260809 16:11:32.424197 25773 raft_consensus.cc:3037] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:32.424908 25773 raft_consensus.cc:504] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.425030 25773 leader_election.cc:288] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [CANDIDATE]: Term 1 election: Requested vote from peers bece2fdc034e46ccbf299b80aff10fd8 (127.16.224.129:39855), 1bdd0b617a74435e9beb09ce0526a186 (127.16.224.130:33629)
I20260809 16:11:32.425145 25243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e2209a35c348ecb0dd22d380c9416f" candidate_uuid: "eb876e5affa24339b412712675222eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bece2fdc034e46ccbf299b80aff10fd8"
I20260809 16:11:32.425203 25457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e2209a35c348ecb0dd22d380c9416f" candidate_uuid: "eb876e5affa24339b412712675222eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bdd0b617a74435e9beb09ce0526a186"
I20260809 16:11:32.425238 25243 raft_consensus.cc:3037] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:32.425271 25457 raft_consensus.cc:3037] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:32.426147 25243 raft_consensus.cc:2454] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb876e5affa24339b412712675222eab in term 1.
I20260809 16:11:32.426147 25457 raft_consensus.cc:2454] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb876e5affa24339b412712675222eab in term 1.
I20260809 16:11:32.426347 25563 leader_election.cc:302] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bece2fdc034e46ccbf299b80aff10fd8, eb876e5affa24339b412712675222eab; no voters:
I20260809 16:11:32.426431 25773 raft_consensus.cc:2781] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:32.426555 25773 raft_consensus.cc:686] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 1 LEADER]: Becoming Leader. State: Replica: eb876e5affa24339b412712675222eab, State: Running, Role: LEADER
I20260809 16:11:32.426631 25773 consensus_queue.cc:237] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } }
I20260809 16:11:32.427438 25005 catalog_manager.cc:5495] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab reported cstate change: term changed from 0 to 1, leader changed from <none> to eb876e5affa24339b412712675222eab (127.16.224.131). New cstate: current_term: 1 leader_uuid: "eb876e5affa24339b412712675222eab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb876e5affa24339b412712675222eab" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:32.437496 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.436730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.16.224.131:42511)
0809 16:11:32.436773 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.436774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.436782 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:32.436847 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.436848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.436858 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.436900 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.437368 (+ 468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.437368 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.437368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.437370 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.437379 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.437381 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.437413 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.437441 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.437450 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:32.439088 25243 raft_consensus.cc:1261] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 1 FOLLOWER]: Refusing update from remote peer eb876e5affa24339b412712675222eab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:32.439097 25457 raft_consensus.cc:1261] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 1 FOLLOWER]: Refusing update from remote peer eb876e5affa24339b412712675222eab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:32.439404 25782 consensus_queue.cc:1035] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bdd0b617a74435e9beb09ce0526a186" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:32.439525 25773 consensus_queue.cc:1035] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bece2fdc034e46ccbf299b80aff10fd8" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 39855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:32.442205 25790 mvcc.cc:205] Tried to move back new op lower bound from 7316651591428292608 to 7316651591380123648. Current Snapshot: MvccSnapshot[applied={T|T < 7316651591428292608}]
I20260809 16:11:32.443660 25795 mvcc.cc:205] Tried to move back new op lower bound from 7316651591428292608 to 7316651591380123648. Current Snapshot: MvccSnapshot[applied={T|T < 7316651591428292608 or (T in {7316651591428292608})}]
I20260809 16:11:32.443958 25003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:32.444046 25003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:32.444761 25003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1196246359474011a48016ae6bb5ec84: Sending DeleteTablet for 3 replicas of tablet c8e2209a35c348ecb0dd22d380c9416f
I20260809 16:11:32.445094 25215 tablet_service.cc:1514] Processing DeleteTablet for tablet c8e2209a35c348ecb0dd22d380c9416f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:32 UTC) from {username='slave'} at 127.0.0.1:42860
I20260809 16:11:32.445099 25427 tablet_service.cc:1514] Processing DeleteTablet for tablet c8e2209a35c348ecb0dd22d380c9416f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:32 UTC) from {username='slave'} at 127.0.0.1:43668
I20260809 16:11:32.445302 25639 tablet_service.cc:1514] Processing DeleteTablet for tablet c8e2209a35c348ecb0dd22d380c9416f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:32 UTC) from {username='slave'} at 127.0.0.1:59368
I20260809 16:11:32.445331 25810 tablet_replica.cc:330] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: stopping tablet replica
I20260809 16:11:32.445427 25810 raft_consensus.cc:2229] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:32.445544 25810 raft_consensus.cc:2258] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:32.445549 25811 tablet_replica.cc:330] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: stopping tablet replica
I20260809 16:11:32.445706 25809 tablet_replica.cc:330] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8: stopping tablet replica
I20260809 16:11:32.445706 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 25102
I20260809 16:11:32.445719 25811 raft_consensus.cc:2229] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:32.445797 25809 raft_consensus.cc:2229] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:32.445883 25809 raft_consensus.cc:2258] T c8e2209a35c348ecb0dd22d380c9416f P bece2fdc034e46ccbf299b80aff10fd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:32.448365 25811 raft_consensus.cc:2258] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:32.454396 25811 ts_tablet_manager.cc:1905] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:32.457558 25811 ts_tablet_manager.cc:1918] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:32.457643 25811 log.cc:1196] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/c8e2209a35c348ecb0dd22d380c9416f
I20260809 16:11:32.457978 25811 ts_tablet_manager.cc:1939] T c8e2209a35c348ecb0dd22d380c9416f P eb876e5affa24339b412712675222eab: Deleting consensus metadata
I20260809 16:11:32.458583 24980 catalog_manager.cc:4841] TS eb876e5affa24339b412712675222eab (127.16.224.131:42511): tablet c8e2209a35c348ecb0dd22d380c9416f (table test-table [id=98d9fc45e6ca47fe9a58fc595f9f166e]) successfully deleted
I20260809 16:11:32.464514 25810 ts_tablet_manager.cc:1905] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:32.466168 25810 ts_tablet_manager.cc:1918] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:32.466249 25810 log.cc:1196] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/c8e2209a35c348ecb0dd22d380c9416f
I20260809 16:11:32.466562 25810 ts_tablet_manager.cc:1939] T c8e2209a35c348ecb0dd22d380c9416f P 1bdd0b617a74435e9beb09ce0526a186: Deleting consensus metadata
I20260809 16:11:32.467027 24985 catalog_manager.cc:4841] TS 1bdd0b617a74435e9beb09ce0526a186 (127.16.224.130:33629): tablet c8e2209a35c348ecb0dd22d380c9416f (table test-table [id=98d9fc45e6ca47fe9a58fc595f9f166e]) successfully deleted
W20260809 16:11:32.468717 24983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:39855 (error 108)
I20260809 16:11:32.468971 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 25321
I20260809 16:11:32.470975 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.469098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:39855 (local address 127.0.0.1:42870)
0809 16:11:32.470869 (+ 1771us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.470935 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:39855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1741}
W20260809 16:11:32.492429 24983 catalog_manager.cc:4584] TS bece2fdc034e46ccbf299b80aff10fd8 (127.16.224.129:39855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8e2209a35c348ecb0dd22d380c9416f: Network error: Client connection negotiation failed: client connection to 127.16.224.129:39855: connect: Connection refused (error 111)
I20260809 16:11:32.496716 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 25537
I20260809 16:11:32.503680 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 24954
2026-08-09T16:11:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (773 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 16:11:32.533591 17282 mini_oidc.cc:152] Starting JWKS server
I20260809 16:11:32.534965 17282 webserver.cc:460] Webserver started at https://127.0.0.1:34667/ using document root <none> and password file <none>
I20260809 16:11:32.535048 17282 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:11:32.535184 17282 webserver.cc:460] Webserver started at http://127.0.0.1:41995/ using document root <none> and password file <none>
2026-08-09T16:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:32Z Disabled control of system clock
I20260809 16:11:32.543571 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40087
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:39913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:40087
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34667/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/testchainca.cert with env {}
W20260809 16:11:32.625033 25820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:32.625257 25820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:32.625278 25820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:32.626441 25820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:32.626488 25820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:32.626499 25820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:32.626510 25820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:32.626521 25820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:11:32.626531 25820 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:11:32.626541 25820 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34667/jwks.json
I20260809 16:11:32.628341 25820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:39913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:40087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34667/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:32.628575 25820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:32.628749 25820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:32.633714 25826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.633888 25820 server_base.cc:1034] running on GCE node
W20260809 16:11:32.633713 25828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:32.633714 25831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.634214 25820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:32.634555 25820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:32.635711 25820 hybrid_clock.cc:648] HybridClock initialized: now 1786291892635679 us; error 51 us; skew 500 ppm
I20260809 16:11:32.636974 25820 webserver.cc:460] Webserver started at http://127.16.224.190:46633/ using document root <none> and password file <none>
I20260809 16:11:32.637194 25820 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:32.637244 25820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:32.637382 25820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:32.638540 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "c16620748623411e90d5578fe7a94e69"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "00647a4183cae4176eaf4d003e6b81a5"
server_key_iv: "b98067cb4c4ff5887bd2f5c229673a7a"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.638888 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "c16620748623411e90d5578fe7a94e69"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "00647a4183cae4176eaf4d003e6b81a5"
server_key_iv: "b98067cb4c4ff5887bd2f5c229673a7a"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.640152 25820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:32.640890 25840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.641047 25820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:32.641137 25820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "c16620748623411e90d5578fe7a94e69"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "00647a4183cae4176eaf4d003e6b81a5"
server_key_iv: "b98067cb4c4ff5887bd2f5c229673a7a"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.641199 25820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:32.651394 25820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:32.651716 25820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:32.651849 25820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:32.655864 25820 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:40087
I20260809 16:11:32.655898 25913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:40087 every 8 connection(s)
I20260809 16:11:32.656221 25820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:32.657155 25914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:32.657780 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 25820
I20260809 16:11:32.657876 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:32.658051 17282 external_mini_cluster.cc:1428] Setting key 2a4e506ba9e0ce3d4485672a1441ab8f
I20260809 16:11:32.660357 25914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69: Bootstrap starting.
I20260809 16:11:32.661080 25914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:32.661389 25914 log.cc:824] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:32.662200 25914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69: No bootstrap required, opened a new log
I20260809 16:11:32.664103 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.658569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46642 (local address 127.16.224.190:40087)
0809 16:11:32.658793 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.658803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.658839 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:11:32.659360 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.659370 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.659453 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.659613 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.659780 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.661109 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.662038 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.662653 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.662786 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:32.663402 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.663406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.663407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.663619 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.663665 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.663679 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.663748 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.663803 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.663888 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260809 16:11:32.664693 25914 raft_consensus.cc:348] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16620748623411e90d5578fe7a94e69" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40087 } }
I20260809 16:11:32.664875 25914 raft_consensus.cc:374] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:32.664925 25914 raft_consensus.cc:729] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c16620748623411e90d5578fe7a94e69, State: Initialized, Role: FOLLOWER
I20260809 16:11:32.665046 25914 consensus_queue.cc:260] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16620748623411e90d5578fe7a94e69" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40087 } }
I20260809 16:11:32.665129 25914 raft_consensus.cc:388] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:32.665136 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:11:32.665158 25914 raft_consensus.cc:482] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:32.665196 25914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:32.665722 25914 raft_consensus.cc:504] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16620748623411e90d5578fe7a94e69" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40087 } }
I20260809 16:11:32.665838 25914 leader_election.cc:302] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c16620748623411e90d5578fe7a94e69; no voters:
I20260809 16:11:32.666002 25914 leader_election.cc:288] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:32.666046 25920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:32.666213 25920 raft_consensus.cc:686] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [term 1 LEADER]: Becoming Leader. State: Replica: c16620748623411e90d5578fe7a94e69, State: Running, Role: LEADER
I20260809 16:11:32.666289 25914 sys_catalog.cc:564] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:32.666303 25920 consensus_queue.cc:237] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16620748623411e90d5578fe7a94e69" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40087 } }
I20260809 16:11:32.666742 25922 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c16620748623411e90d5578fe7a94e69. Latest consensus state: current_term: 1 leader_uuid: "c16620748623411e90d5578fe7a94e69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16620748623411e90d5578fe7a94e69" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40087 } } }
I20260809 16:11:32.666756 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c16620748623411e90d5578fe7a94e69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16620748623411e90d5578fe7a94e69" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40087 } } }
I20260809 16:11:32.666872 25922 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:32.666879 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:32.667078 25929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:32.667693 25929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:32.669082 25929 catalog_manager.cc:1347] Generated new cluster ID: ae36dbfe2d8a449eba57aa7066bf4f03
I20260809 16:11:32.669126 25929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:32.688534 25929 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:32.689071 25929 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:32.695197 25929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c16620748623411e90d5578fe7a94e69: Generated new TSK 0
I20260809 16:11:32.695396 25929 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:11:32.724223 25964 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 16:11:32.724463 25964 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 16:11:32.724913 25961 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:32.709424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:40087 (local address 127.0.0.1:46644)
0809 16:11:32.709610 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.709620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.709691 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.709857 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.709858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.709860 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:11:32.709931 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:32.709933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.710686 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.710687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:32.711315 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:32.711317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.711671 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.711672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:32.711740 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:32.711764 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:11:32.724464 (+ 12700us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:11:32.724585 (+ 121us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.224.190:40087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.16.224.190:40087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 16:11:32.725351 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.709477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.16.224.190:40087)
0809 16:11:32.709522 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.709524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.709678 (+ 154us) server_negotiation.cc:407] Connection header received
0809 16:11:32.709782 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.709783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.709795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.709851 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:11:32.710000 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.710593 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.711393 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.711595 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.711652 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:32.711875 (+ 223us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:11:32.724281 (+ 12406us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:11:32.725156 (+ 875us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:32.725381 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 25820
W20260809 16:11:32.725402 25918 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-08-09T16:11:32Z chronyd exiting
I20260809 16:11:32.791272 17282 test_util.cc:175] -----------------------------------------------
I20260809 16:11:32.791368 17282 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786291868496945-17282-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (258 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 16:11:32.792153 17282 mini_oidc.cc:152] Starting JWKS server
I20260809 16:11:32.792824 17282 webserver.cc:460] Webserver started at https://127.0.0.1:42519/ using document root <none> and password file <none>
I20260809 16:11:32.792865 17282 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:11:32.793015 17282 webserver.cc:460] Webserver started at http://127.0.0.1:37051/ using document root <none> and password file <none>
2026-08-09T16:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:32Z Disabled control of system clock
I20260809 16:11:32.801261 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:36323
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:38617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:36323
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42519/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/testchainca.cert with env {}
W20260809 16:11:32.883896 25976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:32.884121 25976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:32.884155 25976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:32.885334 25976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:32.885383 25976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:32.885403 25976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:32.885422 25976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:32.885442 25976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:11:32.885468 25976 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:11:32.885493 25976 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42519/jwks.json
I20260809 16:11:32.887086 25976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:36323
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:36323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42519/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:32.887347 25976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:32.887584 25976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:32.892320 25985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:32.892355 25988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.892496 25976 server_base.cc:1034] running on GCE node
W20260809 16:11:32.892344 25993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:32.892741 25976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:32.893049 25976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:32.894201 25976 hybrid_clock.cc:648] HybridClock initialized: now 1786291892894184 us; error 29 us; skew 500 ppm
I20260809 16:11:32.895490 25976 webserver.cc:460] Webserver started at http://127.16.224.190:40151/ using document root <none> and password file <none>
I20260809 16:11:32.895717 25976 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:32.895769 25976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:32.895905 25976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:32.896958 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "34056e08894f469fadd6c8ef7f1c72b1"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "5606c116053db2a6e0d98a6522a4d279"
server_key_iv: "ad55731dc0d6cf758ea9affbd418a382"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.897322 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "34056e08894f469fadd6c8ef7f1c72b1"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "5606c116053db2a6e0d98a6522a4d279"
server_key_iv: "ad55731dc0d6cf758ea9affbd418a382"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.898679 25976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:32.899454 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:32.899623 25976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:32.899710 25976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "34056e08894f469fadd6c8ef7f1c72b1"
format_stamp: "Formatted at 2026-08-09 16:11:32 on dist-test-slave-fgb3"
server_key: "5606c116053db2a6e0d98a6522a4d279"
server_key_iv: "ad55731dc0d6cf758ea9affbd418a382"
server_key_version: "encryptionkey@0"
I20260809 16:11:32.899770 25976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:32.915944 25976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:32.916289 25976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:32.916452 25976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:32.920585 25976 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:36323
I20260809 16:11:32.920622 26080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:36323 every 8 connection(s)
I20260809 16:11:32.920964 25976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:32.921653 26083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:32.924975 26083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1: Bootstrap starting.
I20260809 16:11:32.925554 26083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:32.925736 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 25976
I20260809 16:11:32.925850 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:32.925905 26083 log.cc:824] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:32.926056 17282 external_mini_cluster.cc:1428] Setting key 7c2ceb3c2f17988ccaf3a04f088ef853
I20260809 16:11:32.926738 26083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1: No bootstrap required, opened a new log
I20260809 16:11:32.929479 26083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34056e08894f469fadd6c8ef7f1c72b1" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36323 } }
I20260809 16:11:32.929622 26083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:32.929644 26083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34056e08894f469fadd6c8ef7f1c72b1, State: Initialized, Role: FOLLOWER
I20260809 16:11:32.929754 26083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34056e08894f469fadd6c8ef7f1c72b1" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36323 } }
I20260809 16:11:32.929828 26083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:32.929852 26083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:32.929908 26083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:32.930624 26083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34056e08894f469fadd6c8ef7f1c72b1" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36323 } }
I20260809 16:11:32.930739 26083 leader_election.cc:302] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34056e08894f469fadd6c8ef7f1c72b1; no voters:
I20260809 16:11:32.930909 26083 leader_election.cc:288] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:32.930948 26092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:32.931139 26092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [term 1 LEADER]: Becoming Leader. State: Replica: 34056e08894f469fadd6c8ef7f1c72b1, State: Running, Role: LEADER
I20260809 16:11:32.931164 26083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:32.931238 26092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34056e08894f469fadd6c8ef7f1c72b1" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36323 } }
I20260809 16:11:32.931720 26094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34056e08894f469fadd6c8ef7f1c72b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34056e08894f469fadd6c8ef7f1c72b1" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36323 } } }
I20260809 16:11:32.931728 26093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34056e08894f469fadd6c8ef7f1c72b1. Latest consensus state: current_term: 1 leader_uuid: "34056e08894f469fadd6c8ef7f1c72b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34056e08894f469fadd6c8ef7f1c72b1" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36323 } } }
I20260809 16:11:32.931823 26094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:32.931850 26093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:32.932022 26107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:32.932519 26107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:32.932725 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.926580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51980 (local address 127.16.224.190:36323)
0809 16:11:32.926811 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.926820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.926858 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:11:32.927361 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.927384 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.927467 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.927626 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:32.927814 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.929223 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.930133 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.930802 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.930936 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:32.931828 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:32.931833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:32.931834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:32.932050 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:32.932095 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:32.932110 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:32.932174 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:32.932445 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 16:11:32.932526 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260809 16:11:32.933744 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:11:32.934180 26107 catalog_manager.cc:1347] Generated new cluster ID: 8b0a04f9950f4cd18226106588eab335
I20260809 16:11:32.934229 26107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:32.941443 26107 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:32.942135 26107 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:32.948572 26107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34056e08894f469fadd6c8ef7f1c72b1: Generated new TSK 0
I20260809 16:11:32.948741 26107 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:11:32.979223 26134 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 16:11:32.979324 26134 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 16:11:32.979575 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:32.957909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.16.224.190:36323)
0809 16:11:32.957963 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:32.957965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:32.957972 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:32.958013 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:32.958013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:32.958023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:32.958079 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:11:32.958203 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.958722 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.959575 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.959768 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.959821 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:32.959982 (+ 161us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:11:32.979345 (+ 19363us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:11:32.979516 (+ 171us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:11:32.979630 26091 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 16:11:32.979714 26131 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:32.957384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:36323 (local address 127.0.0.1:51996)
0809 16:11:32.957862 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:32.957874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:32.957929 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:32.958085 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:32.958086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:32.958088 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:11:32.958147 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:32.958150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.958811 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.958812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:32.959497 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:32.959500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:32.959829 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:32.959830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:32.959891 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:32.959897 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:11:32.979496 (+ 19599us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:11:32.979615 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.224.190:36323: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":86,"threads_started":1}
I20260809 16:11:32.980023 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 25976
2026-08-09T16:11:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (242 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 16:11:33.035050 17282 mini_oidc.cc:152] Starting JWKS server
I20260809 16:11:33.035733 17282 webserver.cc:460] Webserver started at https://127.0.0.1:41361/ using document root <none> and password file <none>
I20260809 16:11:33.035779 17282 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:11:33.035902 17282 webserver.cc:460] Webserver started at http://127.0.0.1:42615/ using document root <none> and password file <none>
2026-08-09T16:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:33Z Disabled control of system clock
I20260809 16:11:33.044484 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42671
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42671
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41361/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:11:33.124812 26140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:33.125021 26140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:33.125041 26140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:33.126111 26140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:33.126143 26140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:33.126154 26140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:33.126165 26140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:33.126181 26140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:11:33.126192 26140 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:11:33.126207 26140 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41361/jwks.json
I20260809 16:11:33.127535 26140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42671
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41361/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:33.127730 26140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:33.128018 26140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:33.132524 26147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:33.132517 26150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:33.132520 26146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:33.132577 26140 server_base.cc:1034] running on GCE node
I20260809 16:11:33.132928 26140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:33.133184 26140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:33.134385 26140 hybrid_clock.cc:648] HybridClock initialized: now 1786291893134372 us; error 27 us; skew 500 ppm
I20260809 16:11:33.135687 26140 webserver.cc:460] Webserver started at http://127.16.224.190:33537/ using document root <none> and password file <none>
I20260809 16:11:33.135919 26140 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:33.135964 26140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:33.136094 26140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:33.137162 26140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "1900ca51982b473496492a15041ea0c6"
format_stamp: "Formatted at 2026-08-09 16:11:33 on dist-test-slave-fgb3"
server_key: "9158b4a6664c71e185837f754cd83eb8"
server_key_iv: "c5f5e21e163e44f2777dcb38ff828b9c"
server_key_version: "encryptionkey@0"
I20260809 16:11:33.137501 26140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "1900ca51982b473496492a15041ea0c6"
format_stamp: "Formatted at 2026-08-09 16:11:33 on dist-test-slave-fgb3"
server_key: "9158b4a6664c71e185837f754cd83eb8"
server_key_iv: "c5f5e21e163e44f2777dcb38ff828b9c"
server_key_version: "encryptionkey@0"
I20260809 16:11:33.138769 26140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:33.139477 26160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:33.139621 26140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:33.139700 26140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "1900ca51982b473496492a15041ea0c6"
format_stamp: "Formatted at 2026-08-09 16:11:33 on dist-test-slave-fgb3"
server_key: "9158b4a6664c71e185837f754cd83eb8"
server_key_iv: "c5f5e21e163e44f2777dcb38ff828b9c"
server_key_version: "encryptionkey@0"
I20260809 16:11:33.139765 26140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:33.172156 26140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:33.172497 26140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:33.172631 26140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:33.176548 26140 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:42671
I20260809 16:11:33.176587 26256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:42671 every 8 connection(s)
I20260809 16:11:33.176908 26140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:33.177645 26257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:33.178555 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 26140
I20260809 16:11:33.178660 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:33.178853 17282 external_mini_cluster.cc:1428] Setting key bb729e8c4c665bcbafa9555f66f21492
I20260809 16:11:33.180962 26257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6: Bootstrap starting.
I20260809 16:11:33.181588 26257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:33.181902 26257 log.cc:824] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:33.182617 26257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6: No bootstrap required, opened a new log
I20260809 16:11:33.184172 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:33.179405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.16.224.190:42671)
0809 16:11:33.179602 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:33.179609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:33.179635 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:11:33.180060 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:33.180067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:33.180128 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:33.180244 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:33.180483 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.181403 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.182370 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.182817 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.182946 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:33.183574 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:33.183579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:33.183579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:33.183754 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:33.183786 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:33.183797 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:33.183858 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:33.183933 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:11:33.184008 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260809 16:11:33.184420 26257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1900ca51982b473496492a15041ea0c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42671 } }
I20260809 16:11:33.184582 26257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:33.184607 26257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1900ca51982b473496492a15041ea0c6, State: Initialized, Role: FOLLOWER
I20260809 16:11:33.184721 26257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1900ca51982b473496492a15041ea0c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42671 } }
I20260809 16:11:33.184806 26257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:33.184837 26257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:33.184873 26257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:33.185173 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:11:33.185472 26257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1900ca51982b473496492a15041ea0c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42671 } }
I20260809 16:11:33.185572 26257 leader_election.cc:302] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1900ca51982b473496492a15041ea0c6; no voters:
I20260809 16:11:33.185726 26257 leader_election.cc:288] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:33.185774 26263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:33.185966 26257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:33.185992 26263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [term 1 LEADER]: Becoming Leader. State: Replica: 1900ca51982b473496492a15041ea0c6, State: Running, Role: LEADER
I20260809 16:11:33.186084 26263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1900ca51982b473496492a15041ea0c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42671 } }
I20260809 16:11:33.186481 26264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1900ca51982b473496492a15041ea0c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1900ca51982b473496492a15041ea0c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42671 } } }
I20260809 16:11:33.186488 26265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1900ca51982b473496492a15041ea0c6. Latest consensus state: current_term: 1 leader_uuid: "1900ca51982b473496492a15041ea0c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1900ca51982b473496492a15041ea0c6" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42671 } } }
I20260809 16:11:33.186579 26264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:33.186604 26265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:33.186813 26275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:33.187296 26275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:33.188776 26275 catalog_manager.cc:1347] Generated new cluster ID: 0cce7bf9cff942179bf68a95007504da
I20260809 16:11:33.188838 26275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:33.215806 26275 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:33.216398 26275 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:33.219738 26275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1900ca51982b473496492a15041ea0c6: Generated new TSK 0
I20260809 16:11:33.219919 26275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:11:33.233707 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:33.230143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.16.224.190:42671)
0809 16:11:33.230203 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:33.230205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:33.230304 (+ 99us) server_negotiation.cc:407] Connection header received
0809 16:11:33.230400 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:33.230402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:33.230412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:33.230470 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:11:33.230606 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.231080 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.232213 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.232408 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.232448 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:33.232624 (+ 176us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:11:33.233498 (+ 874us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:11:33.233631 (+ 133us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 16:11:33.233770 26261 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 16:11:33.233969 26297 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:33.230075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42671 (local address 127.0.0.1:38836)
0809 16:11:33.230236 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:33.230247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:33.230335 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:33.230487 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:33.230488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:33.230490 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:11:33.230553 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:33.230556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.231168 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.231170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:33.232138 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:33.232141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.232479 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.232480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:33.232541 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:33.232546 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:11:33.233600 (+ 1054us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:11:33.233706 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.224.190:42671: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260809 16:11:33.234167 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 26140
E20260809 16:11:33.261360 26298 webserver.cc:533] Webserver: sslize failed:
E20260809 16:11:33.261439 26298 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T16:11:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (256 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T16:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:33Z Disabled control of system clock
I20260809 16:11:33.299626 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:36387
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:33165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:36387
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/testchainca.cert with env {}
W20260809 16:11:33.384361 26305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:33.384580 26305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:33.384601 26305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:33.385746 26305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:33.385782 26305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:33.385793 26305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:33.385804 26305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:33.385815 26305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:11:33.385825 26305 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:11:33.385835 26305 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 16:11:33.387362 26305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:33165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:36387
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:36387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:33.387583 26305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:33.387756 26305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:33.392540 26320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:33.392547 26316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:33.392568 26314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:33.392745 26305 server_base.cc:1034] running on GCE node
I20260809 16:11:33.392975 26305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:33.393257 26305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:33.394371 26305 hybrid_clock.cc:648] HybridClock initialized: now 1786291893394352 us; error 28 us; skew 500 ppm
I20260809 16:11:33.395577 26305 webserver.cc:460] Webserver started at http://127.16.224.190:44285/ using document root <none> and password file <none>
I20260809 16:11:33.395787 26305 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:33.395835 26305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:33.395964 26305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:33.397009 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "23622f2dd136448290bf2cda13852c11"
format_stamp: "Formatted at 2026-08-09 16:11:33 on dist-test-slave-fgb3"
server_key: "f609d6e1e8c30362bf191a1bccfa9c1e"
server_key_iv: "c8758ac82a32508174e9d98b12e73de6"
server_key_version: "encryptionkey@0"
I20260809 16:11:33.397349 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "23622f2dd136448290bf2cda13852c11"
format_stamp: "Formatted at 2026-08-09 16:11:33 on dist-test-slave-fgb3"
server_key: "f609d6e1e8c30362bf191a1bccfa9c1e"
server_key_iv: "c8758ac82a32508174e9d98b12e73de6"
server_key_version: "encryptionkey@0"
I20260809 16:11:33.398612 26305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:33.399277 26327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:33.399427 26305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:33.399519 26305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "23622f2dd136448290bf2cda13852c11"
format_stamp: "Formatted at 2026-08-09 16:11:33 on dist-test-slave-fgb3"
server_key: "f609d6e1e8c30362bf191a1bccfa9c1e"
server_key_iv: "c8758ac82a32508174e9d98b12e73de6"
server_key_version: "encryptionkey@0"
I20260809 16:11:33.399580 26305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:33.412618 26305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:33.412945 26305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:33.413080 26305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:33.417124 26305 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:36387
I20260809 16:11:33.417163 26414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:36387 every 8 connection(s)
I20260809 16:11:33.417515 26305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:33.418553 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:33.420946 26415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11: Bootstrap starting.
I20260809 16:11:33.421514 26415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:33.421769 26415 log.cc:824] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:33.422448 26415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11: No bootstrap required, opened a new log
I20260809 16:11:33.423990 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 26305
I20260809 16:11:33.424091 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:33.424278 17282 external_mini_cluster.cc:1428] Setting key dc23fccbc2e9294895333031e6d0b634
I20260809 16:11:33.424137 26415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23622f2dd136448290bf2cda13852c11" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36387 } }
I20260809 16:11:33.424312 26415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:33.424356 26415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23622f2dd136448290bf2cda13852c11, State: Initialized, Role: FOLLOWER
I20260809 16:11:33.424491 26415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23622f2dd136448290bf2cda13852c11" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36387 } }
I20260809 16:11:33.424558 26415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:33.424587 26415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:33.424631 26415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:33.425431 26415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23622f2dd136448290bf2cda13852c11" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36387 } }
I20260809 16:11:33.425546 26415 leader_election.cc:302] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23622f2dd136448290bf2cda13852c11; no voters:
I20260809 16:11:33.425710 26415 leader_election.cc:288] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:33.425868 26422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:33.425982 26422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [term 1 LEADER]: Becoming Leader. State: Replica: 23622f2dd136448290bf2cda13852c11, State: Running, Role: LEADER
I20260809 16:11:33.425985 26415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:33.426079 26422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23622f2dd136448290bf2cda13852c11" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36387 } }
I20260809 16:11:33.426508 26423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23622f2dd136448290bf2cda13852c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23622f2dd136448290bf2cda13852c11" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36387 } } }
I20260809 16:11:33.426597 26423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:33.426740 26426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23622f2dd136448290bf2cda13852c11. Latest consensus state: current_term: 1 leader_uuid: "23622f2dd136448290bf2cda13852c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23622f2dd136448290bf2cda13852c11" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36387 } } }
I20260809 16:11:33.426805 26426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:33.426923 26435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:33.427457 26435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:33.429395 26435 catalog_manager.cc:1347] Generated new cluster ID: dad5ab95f3dc4a3cb470f37687f7e3a7
I20260809 16:11:33.429451 26435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:33.429941 26421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:33.424728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.16.224.190:36387)
0809 16:11:33.424955 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:33.424964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:33.424999 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:11:33.425115 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:33.425120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:33.425181 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:33.425287 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:33.425542 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.426820 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.427883 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.428494 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.428617 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:33.429243 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:33.429247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:33.429248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:11:33.429458 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:33.429503 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:11:33.429517 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:33.429583 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:33.429671 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 16:11:33.429747 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260809 16:11:33.430830 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:11:33.446442 26435 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:33.446988 26435 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:33.452685 26435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23622f2dd136448290bf2cda13852c11: Generated new TSK 0
I20260809 16:11:33.452857 26435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:11:33.884635 26421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:33.458017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.16.224.190:36387)
0809 16:11:33.458073 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:33.458075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:33.458165 (+ 90us) server_negotiation.cc:407] Connection header received
0809 16:11:33.458253 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:33.458254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:33.458265 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:33.458361 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:11:33.458496 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.459023 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.460185 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.460388 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.460441 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:33.460609 (+ 168us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:11:33.884398 (+423789us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:11:33.884569 (+ 171us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55052: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 16:11:33.884706 26421 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55052: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 16:11:33.884811 26464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:33.457928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:36387 (local address 127.0.0.1:55052)
0809 16:11:33.458103 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:33.458114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:33.458185 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:33.458368 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:33.458369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:33.458371 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:11:33.458435 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:33.458437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.459099 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.459100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:33.460090 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:33.460094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:33.460461 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:33.460463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:33.460522 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:33.460528 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:11:33.884562 (+424034us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:11:33.884701 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.224.190:36387: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260809 16:11:33.885191 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 26305
2026-08-09T16:11:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (620 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:34 dist-test-slave-fgb3 krb5kdc[26487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:34 dist-test-slave-fgb3 krb5kdc[26487](info): set up 2 sockets
Aug 09 16:11:34 dist-test-slave-fgb3 krb5kdc[26487](info): commencing operation
krb5kdc: starting...
W20260809 16:11:35.198472 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:35.312884 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:32997
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:33181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:32997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:35.395586 26515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:35.395802 26515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:35.395830 26515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:35.396967 26515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:35.397008 26515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:35.397022 26515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:35.397032 26515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:35.397043 26515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:35.398579 26515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:33181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:32997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:32997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:35.398802 26515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:35.399029 26515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:35.403806 26525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.403847 26531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.403831 26529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:35.403911 26515 server_base.cc:1034] running on GCE node
I20260809 16:11:35.404276 26515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:35.404587 26515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:35.405752 26515 hybrid_clock.cc:648] HybridClock initialized: now 1786291895405728 us; error 26 us; skew 500 ppm
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:35.408236 26515 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:35.408679 26515 webserver.cc:460] Webserver started at http://127.16.224.190:40573/ using document root <none> and password file <none>
I20260809 16:11:35.408918 26515 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:35.408973 26515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:35.409112 26515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:35.410243 26515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "fe077c42cf514b62aafdb326136624d2"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "9429c163224209e3911884d7cd3dd680"
server_key_iv: "3914cb13ddd0b78ed4044b125018e616"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.410672 26515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "fe077c42cf514b62aafdb326136624d2"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "9429c163224209e3911884d7cd3dd680"
server_key_iv: "3914cb13ddd0b78ed4044b125018e616"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.412075 26515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:35.412932 26541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.413106 26515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:35.413204 26515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "fe077c42cf514b62aafdb326136624d2"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "9429c163224209e3911884d7cd3dd680"
server_key_iv: "3914cb13ddd0b78ed4044b125018e616"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.413266 26515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:35.426070 26515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:35.427929 26515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:35.428071 26515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:35.432004 26515 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:32997
I20260809 16:11:35.432040 26630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:32997 every 8 connection(s)
I20260809 16:11:35.432384 26515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:35.433215 26631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:35.435527 26631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2: Bootstrap starting.
I20260809 16:11:35.436074 26631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:35.436311 26631 log.cc:824] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:35.437431 26631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2: No bootstrap required, opened a new log
I20260809 16:11:35.438077 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 26515
I20260809 16:11:35.438180 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:35.438413 17282 external_mini_cluster.cc:1428] Setting key be03eb49086823c9bb32aefde717fcaa
I20260809 16:11:35.439512 26631 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe077c42cf514b62aafdb326136624d2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32997 } }
I20260809 16:11:35.439652 26631 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:35.439683 26631 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe077c42cf514b62aafdb326136624d2, State: Initialized, Role: FOLLOWER
I20260809 16:11:35.439833 26631 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe077c42cf514b62aafdb326136624d2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32997 } }
I20260809 16:11:35.439903 26631 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:35.439934 26631 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:35.439983 26631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:35.440996 26631 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe077c42cf514b62aafdb326136624d2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32997 } }
I20260809 16:11:35.441102 26631 leader_election.cc:302] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe077c42cf514b62aafdb326136624d2; no voters:
I20260809 16:11:35.441250 26631 leader_election.cc:288] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:35.441301 26640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:35.441524 26640 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [term 1 LEADER]: Becoming Leader. State: Replica: fe077c42cf514b62aafdb326136624d2, State: Running, Role: LEADER
I20260809 16:11:35.441504 26631 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:35.441659 26640 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe077c42cf514b62aafdb326136624d2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32997 } }
I20260809 16:11:35.442013 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe077c42cf514b62aafdb326136624d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe077c42cf514b62aafdb326136624d2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32997 } } }
I20260809 16:11:35.442088 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:35.442246 26643 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe077c42cf514b62aafdb326136624d2. Latest consensus state: current_term: 1 leader_uuid: "fe077c42cf514b62aafdb326136624d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe077c42cf514b62aafdb326136624d2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32997 } } }
I20260809 16:11:35.442306 26643 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2 [sys.catalog]: This master's current role is: LEADER
W20260809 16:11:35.443791 26665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:11:35.443852 26665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:11:35.443900 26655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:35.444052 26655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:35.445624 26655 catalog_manager.cc:1347] Generated new cluster ID: 502d7473b9604dd2bf35946d23dad5c4
I20260809 16:11:35.445681 26655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:35.448628 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:35.438936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.16.224.190:32997)
0809 16:11:35.439113 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:35.439124 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:35.439158 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:11:35.439480 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:35.439487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:35.439532 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:35.439652 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:35.440371 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.441372 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.442460 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.442982 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.443206 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:35.445491 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:35.445495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:35.445495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:35.445508 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:35.447086 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.447397 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.447400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.447401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.447515 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.447688 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.447689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.447689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.447924 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:35.448084 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:35.448150 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:11:35.448284 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260809 16:11:35.467988 26655 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:35.468706 26655 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:35.482539 26655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe077c42cf514b62aafdb326136624d2: Generated new TSK 0
I20260809 16:11:35.482728 26655 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:35.511023 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:32997
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:33181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:35.623699 26672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:35.623981 26672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:35.624014 26672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:35.625828 26672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:35.625886 26672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:35.625926 26672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:35.628130 26672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:33181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:35.628399 26672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:35.628618 26672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:35.633512 26679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.633531 26681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.633519 26685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:35.633735 26672 server_base.cc:1034] running on GCE node
I20260809 16:11:35.633921 26672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:35.634148 26672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:35.635285 26672 hybrid_clock.cc:648] HybridClock initialized: now 1786291895635271 us; error 32 us; skew 500 ppm
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:35.637674 26672 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:35.638091 26672 webserver.cc:460] Webserver started at http://127.16.224.129:39063/ using document root <none> and password file <none>
I20260809 16:11:35.638288 26672 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:35.638352 26672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:35.638481 26672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:35.639519 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "17014dd0d738445a96fe6c62dd2ac1ea"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "1476591c4707e02de1b1d6d295878c67"
server_key_iv: "41f33d9e046361c7107dd3cce88eba63"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.639849 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "17014dd0d738445a96fe6c62dd2ac1ea"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "1476591c4707e02de1b1d6d295878c67"
server_key_iv: "41f33d9e046361c7107dd3cce88eba63"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.641105 26672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:35.641798 26704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.641950 26672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:35.642055 26672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "17014dd0d738445a96fe6c62dd2ac1ea"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "1476591c4707e02de1b1d6d295878c67"
server_key_iv: "41f33d9e046361c7107dd3cce88eba63"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.642125 26672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:35.652944 26672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:35.654721 26672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:35.654860 26672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:35.655117 26672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:35.655495 26672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:35.655539 26672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.655567 26672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:35.655591 26672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.661761 26672 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:35117
I20260809 16:11:35.661834 26882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:35117 every 8 connection(s)
I20260809 16:11:35.662124 26672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:35.666971 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 26672
I20260809 16:11:35.667068 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:35.667240 17282 external_mini_cluster.cc:1428] Setting key 3e5c73366d2dca07cb9bfcf8bfada64d
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:35.671319 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:35.662888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:39439 (local address 127.16.224.190:32997)
0809 16:11:35.662954 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:35.662956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:35.663131 (+ 175us) server_negotiation.cc:407] Connection header received
0809 16:11:35.664073 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:35.664075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:35.664090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:35.664129 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:35.664703 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.665204 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.666136 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.666369 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.666426 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:35.668610 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:35.668612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:35.668612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:35.668619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:35.670012 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.670428 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.670429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.670429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.670503 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.670706 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.670707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.670708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.670828 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:35.670890 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:35.671181 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 16:11:35.671253 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:11:35.671557 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:35.662843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:32997 (local address 127.16.224.129:39439)
0809 16:11:35.663005 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:35.663048 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:35.663968 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:35.664189 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:35.664193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:35.664446 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:35.664640 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:35.664645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.665266 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.665267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:35.666076 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:35.666078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.666434 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.666435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:35.666537 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:35.667012 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:35.667022 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:35.668519 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:35.670103 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:35.670110 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:35.670112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:35.670362 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:35.670559 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:35.670559 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:35.670559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:35.670655 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:35.670889 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:35.670896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:35.671052 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 16:11:35.671160 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:11:35.671337 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260809 16:11:35.671999 26883 heartbeater.cc:347] Connected to a master server at 127.16.224.190:32997
I20260809 16:11:35.672099 26883 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:35.672276 26883 heartbeater.cc:511] Master 127.16.224.190:32997 requested a full tablet report, sending...
I20260809 16:11:35.672801 26568 ts_manager.cc:194] Registered new tserver with Master: 17014dd0d738445a96fe6c62dd2ac1ea (127.16.224.129:35117)
I20260809 16:11:35.673389 26568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:39439
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:35.692939 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:32997
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:33181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:35.774664 26894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:35.774870 26894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:35.774893 26894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:35.776032 26894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:35.776077 26894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:35.776101 26894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:35.777593 26894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:33181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:35.777853 26894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:35.778082 26894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:35.782979 26900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.783083 26903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.782975 26901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:35.783285 26894 server_base.cc:1034] running on GCE node
I20260809 16:11:35.783556 26894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:35.783823 26894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:35.784973 26894 hybrid_clock.cc:648] HybridClock initialized: now 1786291895784961 us; error 30 us; skew 500 ppm
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:35.787398 26894 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:35.788008 26894 webserver.cc:460] Webserver started at http://127.16.224.130:34133/ using document root <none> and password file <none>
I20260809 16:11:35.788285 26894 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:35.788343 26894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:35.788482 26894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:35.789880 26894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "f6ce541c6d41449cb706f8f03753e1c4"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "64ced5ac68013d259fa4b586d950dff7"
server_key_iv: "ab670c78e166d31dd896982121118b5d"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.790335 26894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "f6ce541c6d41449cb706f8f03753e1c4"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "64ced5ac68013d259fa4b586d950dff7"
server_key_iv: "ab670c78e166d31dd896982121118b5d"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.792062 26894 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:35.792950 26916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.793119 26894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:35.793221 26894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "f6ce541c6d41449cb706f8f03753e1c4"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "64ced5ac68013d259fa4b586d950dff7"
server_key_iv: "ab670c78e166d31dd896982121118b5d"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.793296 26894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:35.813268 26894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:35.815153 26894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:35.815279 26894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:35.815513 26894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:35.815814 26894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:35.815850 26894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.815883 26894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:35.815899 26894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.821501 26894 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:45771
I20260809 16:11:35.821622 27118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:45771 every 8 connection(s)
I20260809 16:11:35.821877 26894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:35.828402 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 26894
I20260809 16:11:35.828511 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:35.828680 17282 external_mini_cluster.cc:1428] Setting key 4ee4ff86422b170fb58e9facf37af5dd
I20260809 16:11:35.831264 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:35.822609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:60307 (local address 127.16.224.190:32997)
0809 16:11:35.822652 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:35.822654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:35.822845 (+ 191us) server_negotiation.cc:407] Connection header received
0809 16:11:35.823794 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:35.823796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:35.823809 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:35.823862 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:35.824497 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.825001 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.825999 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.826209 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.826269 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:35.828500 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:35.828502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:35.828502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:35.828511 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:35.829937 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.830351 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.830352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.830352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.830418 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.830640 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.830641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.830641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.830759 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:35.830816 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:35.831087 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 16:11:35.831173 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:11:35.831429 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:35.822575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:32997 (local address 127.16.224.130:60307)
0809 16:11:35.822725 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:35.822763 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:35.823686 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:35.823944 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:35.823948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:35.824203 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:35.824402 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:35.824407 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.825089 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.825090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:35.825913 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:35.825915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.826283 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.826284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:35.826424 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:35.826833 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:35.826841 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:35.828385 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:35.830035 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:35.830042 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:35.830044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:35.830260 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:35.830475 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:35.830475 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:35.830476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:35.830583 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:35.830814 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:35.830821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:35.830965 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:11:35.831055 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:11:35.831212 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260809 16:11:35.831881 27120 heartbeater.cc:347] Connected to a master server at 127.16.224.190:32997
I20260809 16:11:35.831992 27120 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:35.832185 27120 heartbeater.cc:511] Master 127.16.224.190:32997 requested a full tablet report, sending...
I20260809 16:11:35.832587 26568 ts_manager.cc:194] Registered new tserver with Master: f6ce541c6d41449cb706f8f03753e1c4 (127.16.224.130:45771)
I20260809 16:11:35.832963 26568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:60307
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:35.853664 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:32997
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:33181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:35.933696 27127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:35.933904 27127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:35.933926 27127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:35.935101 27127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:35.935148 27127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:35.935173 27127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:35.936666 27127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:33181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:35.936903 27127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:35.937137 27127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:35.941994 27145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.942088 27140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:35.942001 27139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:35.942411 27127 server_base.cc:1034] running on GCE node
I20260809 16:11:35.942652 27127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:35.942895 27127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:35.944036 27127 hybrid_clock.cc:648] HybridClock initialized: now 1786291895944012 us; error 34 us; skew 500 ppm
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:35.946589 27127 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:35.947034 27127 webserver.cc:460] Webserver started at http://127.16.224.131:32795/ using document root <none> and password file <none>
I20260809 16:11:35.947240 27127 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:35.947288 27127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:35.947418 27127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:35.948489 27127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "0e8d59db39bb47c189203842070c78fe"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "e7e2a62448500508809edc26c75d65c4"
server_key_iv: "efbd2abbea8411092f49e86e03e54739"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.948843 27127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "0e8d59db39bb47c189203842070c78fe"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "e7e2a62448500508809edc26c75d65c4"
server_key_iv: "efbd2abbea8411092f49e86e03e54739"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.950191 27127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:11:35.950964 27154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.951125 27127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.951211 27127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "0e8d59db39bb47c189203842070c78fe"
format_stamp: "Formatted at 2026-08-09 16:11:35 on dist-test-slave-fgb3"
server_key: "e7e2a62448500508809edc26c75d65c4"
server_key_iv: "efbd2abbea8411092f49e86e03e54739"
server_key_version: "encryptionkey@0"
I20260809 16:11:35.951279 27127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:35.961828 27127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:35.963914 27127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:35.964030 27127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:35.964205 27127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:35.964468 27127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:35.964498 27127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.964519 27127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:35.964533 27127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:35.969954 27127 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:39137
I20260809 16:11:35.970016 27335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:39137 every 8 connection(s)
I20260809 16:11:35.970299 27127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:35.978272 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 27127
I20260809 16:11:35.978379 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:35.978569 17282 external_mini_cluster.cc:1428] Setting key cdc88c0e627a2f22aab4f60ced774fee
I20260809 16:11:35.978763 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:35.971043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:46601 (local address 127.16.224.190:32997)
0809 16:11:35.971105 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:35.971107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:35.971348 (+ 241us) server_negotiation.cc:407] Connection header received
0809 16:11:35.972219 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:35.972220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:35.972232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:35.972293 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:35.972864 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.973361 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.974278 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.974527 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.974573 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:35.976421 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:35.976422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:35.976423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:35.976430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:35.977571 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.977905 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.977905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.977905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.977949 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:35.978128 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:35.978128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:35.978129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:35.978224 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:35.978268 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:35.978599 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 16:11:35.978680 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:11:35.978935 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:35.970992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:32997 (local address 127.16.224.131:46601)
0809 16:11:35.971173 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:35.971214 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:35.972109 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:35.972363 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:35.972366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:35.972616 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:35.972800 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:35.972805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.973421 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.973422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:35.974217 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:35.974219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:35.974585 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:35.974586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:35.974685 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:35.975071 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:35.975079 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:35.976335 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:35.977644 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:35.977650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:35.977652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:35.977853 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:35.977991 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:35.977991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:35.977991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:35.978085 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:35.978274 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:35.978280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:35.978464 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 16:11:35.978578 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:11:35.978738 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260809 16:11:35.979406 27336 heartbeater.cc:347] Connected to a master server at 127.16.224.190:32997
I20260809 16:11:35.979517 27336 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:35.979696 27336 heartbeater.cc:511] Master 127.16.224.190:32997 requested a full tablet report, sending...
I20260809 16:11:35.980057 26568 ts_manager.cc:194] Registered new tserver with Master: 0e8d59db39bb47c189203842070c78fe (127.16.224.131:39137)
I20260809 16:11:35.980412 26568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:46601
I20260809 16:11:35.990125 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/test-admin.keytab.
Aug 09 16:11:35 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:36.005816 17282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 16:11:36.007359 27347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:36.006631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.16.224.129:35117)
0809 16:11:36.006792 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:36.006797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:36.006805 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:36.006836 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:36.006838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:36.006852 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:36.006920 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:36.007275 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43458: BlockingRecv error: recv got EOF from 127.0.0.1:43458 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
W20260809 16:11:36.007431 17282 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:36.008349 17282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:36.009152 17282 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:36.016129 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:36.009649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.16.224.129:35117)
0809 16:11:36.009689 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:36.009691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:36.009701 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:36.009760 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:36.009761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:36.009770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:36.009817 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:36.010136 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:36.010898 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:36.011605 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:36.011907 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:36.011955 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:36.013805 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:36.013814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:36.013814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:36.013820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:36.015245 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:36.015558 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:36.015560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:36.015561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:36.015606 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:36.015739 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:36.015739 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:36.015740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:36.015849 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:36.015939 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:36.016002 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 16:11:36.016061 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:11:36.017765 27347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:36.017264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.16.224.129:35117)
0809 16:11:36.017296 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:36.017297 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:36.017350 (+ 53us) server_negotiation.cc:407] Connection header received
0809 16:11:36.017394 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:36.017395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:36.017402 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:36.017452 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:36.017724 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43484: BlockingRecv error: recv got EOF from 127.0.0.1:43484 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 16:11:36.017810 17282 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:36.018677 17282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:36.019466 17282 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:36.025401 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:36.019953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.16.224.129:35117)
0809 16:11:36.019999 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:36.020000 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:36.020023 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:11:36.020081 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:36.020081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:36.020089 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:36.020132 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:36.020467 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:36.020933 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:36.021583 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:36.021773 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:36.021824 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:36.023660 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:36.023661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:36.023661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:36.023667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:36.024657 (+ 990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:36.024929 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:36.024930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:36.024930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:36.024972 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:36.025089 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:36.025089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:36.025089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:36.025164 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:36.025222 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:36.025293 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:11:36.025345 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 16:11:36.026858 27347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:36.026243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.16.224.129:35117)
0809 16:11:36.026290 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:36.026292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:36.026330 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:11:36.026397 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:36.026398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:36.026406 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:36.026458 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:36.026816 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43500: BlockingRecv error: recv got EOF from 127.0.0.1:43500 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 16:11:36.026934 17282 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786291868496945-17282-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:36.027745 17282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:36.028445 17282 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.129@KRBTEST.COM
I20260809 16:11:36.034715 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:36.028893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43510 (local address 127.16.224.129:35117)
0809 16:11:36.028937 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:36.028939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:36.028980 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:11:36.029020 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:36.029021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:36.029028 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:36.029070 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:36.029411 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:36.029837 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:36.030587 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:36.030756 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:36.030800 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:36.032647 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:36.032649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:36.032649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:36.032656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:36.033773 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:36.034064 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:36.034065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:36.034065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:36.034111 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:36.034267 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:36.034268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:36.034268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:36.034383 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:36.034468 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:36.034564 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:11:36.034630 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:11:36.035269 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 26672
I20260809 16:11:36.055694 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 26894
I20260809 16:11:36.066828 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 27127
I20260809 16:11:36.079829 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 26515
2026-08-09T16:11:36Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2010 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[27366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[27366](info): set up 2 sockets
Aug 09 16:11:36 dist-test-slave-fgb3 krb5kdc[27366](info): commencing operation
krb5kdc: starting...
W20260809 16:11:37.224059 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab.
Entry for principal oryx/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab.
I20260809 16:11:37.341115 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42645
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab
--principal=oryx/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:37.421856 27393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:37.422046 27393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:37.422075 27393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:37.423180 27393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:37.423221 27393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:37.423240 27393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:37.423255 27393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:37.423280 27393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:37.424711 27393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab
--principal=oryx/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:37.424942 27393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:37.425135 27393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:37.429661 27403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:37.429695 27400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:37.429685 27399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:37.429713 27393 server_base.cc:1034] running on GCE node
I20260809 16:11:37.430060 27393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:37.430362 27393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:37.431512 27393 hybrid_clock.cc:648] HybridClock initialized: now 1786291897431494 us; error 34 us; skew 500 ppm
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:37.433853 27393 init.cc:377] Logged in from keytab as oryx/127.16.224.190@KRBTEST.COM (short username oryx)
I20260809 16:11:37.434276 27393 webserver.cc:460] Webserver started at http://127.16.224.190:34025/ using document root <none> and password file <none>
I20260809 16:11:37.434507 27393 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:37.434557 27393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:37.434677 27393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:37.435714 27393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "d4cb60585dc14321b9637dcc9412ddf8"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "e5f7a4132d14a553b9195a2edd7c4368"
server_key_iv: "f0cda32a7acf206300ad357f5fe99bfa"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.436053 27393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "d4cb60585dc14321b9637dcc9412ddf8"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "e5f7a4132d14a553b9195a2edd7c4368"
server_key_iv: "f0cda32a7acf206300ad357f5fe99bfa"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.437291 27393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:11:37.437961 27418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.438143 27393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:37.438230 27393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "d4cb60585dc14321b9637dcc9412ddf8"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "e5f7a4132d14a553b9195a2edd7c4368"
server_key_iv: "f0cda32a7acf206300ad357f5fe99bfa"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.438290 27393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:37.474953 27393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:37.476990 27393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:37.477130 27393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:37.481405 27393 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:42645
I20260809 16:11:37.481453 27504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:42645 every 8 connection(s)
I20260809 16:11:37.481803 27393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:37.482851 27506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:37.485100 27506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: Bootstrap starting.
I20260809 16:11:37.485634 27506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:37.485890 27506 log.cc:824] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:37.486482 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 27393
I20260809 16:11:37.486588 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:37.486655 27506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: No bootstrap required, opened a new log
I20260809 16:11:37.486794 17282 external_mini_cluster.cc:1428] Setting key cfdd8e39073e8f7993337004f7566942
I20260809 16:11:37.488950 27506 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } }
I20260809 16:11:37.489086 27506 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:37.489106 27506 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4cb60585dc14321b9637dcc9412ddf8, State: Initialized, Role: FOLLOWER
I20260809 16:11:37.489212 27506 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } }
I20260809 16:11:37.489281 27506 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:37.489306 27506 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:37.489351 27506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:37.489864 27506 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } }
I20260809 16:11:37.489953 27506 leader_election.cc:302] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4cb60585dc14321b9637dcc9412ddf8; no voters:
I20260809 16:11:37.490103 27506 leader_election.cc:288] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:37.490139 27512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:37.490278 27512 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 1 LEADER]: Becoming Leader. State: Replica: d4cb60585dc14321b9637dcc9412ddf8, State: Running, Role: LEADER
I20260809 16:11:37.490392 27512 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } }
I20260809 16:11:37.490445 27506 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:37.490803 27515 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4cb60585dc14321b9637dcc9412ddf8. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } }
I20260809 16:11:37.490804 27512 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } }
I20260809 16:11:37.490903 27515 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:37.490917 27512 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
W20260809 16:11:37.492172 27542 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:11:37.492223 27542 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:11:37.492269 27530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:37.492426 27530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:37.493947 27530 catalog_manager.cc:1347] Generated new cluster ID: 4743acfe20e34f9795156e1abdc3e193
I20260809 16:11:37.493999 27530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:37.497614 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:37.487305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.16.224.190:42645)
0809 16:11:37.487493 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:37.487502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:37.487528 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:11:37.487981 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:37.487990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:37.488054 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:37.488180 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:37.488895 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.490164 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.491232 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.491881 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.492016 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:37.494687 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:37.494691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:37.494692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:37.494702 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:37.496364 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.496686 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.496688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.496689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.496771 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.496911 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.496912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.496912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.497117 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:37.497211 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:37.497317 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 16:11:37.497419 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260809 16:11:37.499843 27530 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:37.500473 27530 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:37.512876 27530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: Generated new TSK 0
I20260809 16:11:37.513049 27530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab.
Entry for principal oryx/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab.
I20260809 16:11:37.540609 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42645
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab
--principal=oryx/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:37.622046 27549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:37.622252 27549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:37.622282 27549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:37.623436 27549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:37.623479 27549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:37.623504 27549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:37.624969 27549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab
--principal=oryx/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:37.625192 27549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:37.625425 27549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:37.630223 27560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:37.630229 27557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:37.630234 27554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:37.630455 27549 server_base.cc:1034] running on GCE node
I20260809 16:11:37.630643 27549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:37.630867 27549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:37.632032 27549 hybrid_clock.cc:648] HybridClock initialized: now 1786291897632011 us; error 34 us; skew 500 ppm
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:37.634667 27549 init.cc:377] Logged in from keytab as oryx/127.16.224.129@KRBTEST.COM (short username oryx)
I20260809 16:11:37.635121 27549 webserver.cc:460] Webserver started at http://127.16.224.129:43035/ using document root <none> and password file <none>
I20260809 16:11:37.635356 27549 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:37.635404 27549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:37.635535 27549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:37.636595 27549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "ebba9f24fa6a45d89db719d5530fb661"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "5ff205bb9853f30eb5c995aebe04384c"
server_key_iv: "a227d894d1c42241712290ef4d44b77c"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.636929 27549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "ebba9f24fa6a45d89db719d5530fb661"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "5ff205bb9853f30eb5c995aebe04384c"
server_key_iv: "a227d894d1c42241712290ef4d44b77c"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.638186 27549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:37.638937 27577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.639081 27549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.639169 27549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "ebba9f24fa6a45d89db719d5530fb661"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "5ff205bb9853f30eb5c995aebe04384c"
server_key_iv: "a227d894d1c42241712290ef4d44b77c"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.639238 27549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:37.655148 27549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:37.656932 27549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:37.657068 27549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:37.657289 27549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:37.657588 27549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:37.657622 27549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.657652 27549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:37.657673 27549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.663422 27549 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:41065
I20260809 16:11:37.663487 27762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:41065 every 8 connection(s)
I20260809 16:11:37.663846 27549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:37.665308 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 27549
I20260809 16:11:37.665409 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:37.665612 17282 external_mini_cluster.cc:1428] Setting key 75d82f91b279d9249fe3bf84942e1266
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:37.674216 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:37.665457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:56189 (local address 127.16.224.190:42645)
0809 16:11:37.665506 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:37.665508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:37.665793 (+ 285us) server_negotiation.cc:407] Connection header received
0809 16:11:37.666294 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:37.666295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:37.666306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:37.666371 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:37.667577 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.668121 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.669114 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.669345 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.669401 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:37.671583 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:37.671584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:37.671585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:37.671593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:37.672993 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.673351 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.673353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.673353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.673414 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.673615 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.673615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.673616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.673746 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:37.673816 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:37.674037 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 16:11:37.674120 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:11:37.674270 27767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:37.665451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.129:56189)
0809 16:11:37.665643 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:37.665699 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:37.666205 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:37.666458 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:37.666464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:37.667199 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:37.667489 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:37.667495 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.668220 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.668221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:37.669030 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:37.669033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.669422 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.669423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:37.669560 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:37.669994 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:37.670003 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:37.671484 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:37.673069 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:37.673075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:37.673077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:37.673285 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:37.673472 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:37.673473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:37.673473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:37.673564 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:37.673801 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:37.673804 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:37.673921 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 16:11:37.673996 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:11:37.674111 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260809 16:11:37.674738 27763 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42645
I20260809 16:11:37.674835 27763 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:37.675004 27763 heartbeater.cc:511] Master 127.16.224.190:42645 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.16.224.130@KRBTEST.COM; defaulting to no policy
I20260809 16:11:37.675549 27441 ts_manager.cc:194] Registered new tserver with Master: ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129:41065)
I20260809 16:11:37.676137 27441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:56189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab.
Entry for principal oryx/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab.
I20260809 16:11:37.691213 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42645
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab
--principal=oryx/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:37.772907 27772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:37.773103 27772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:37.773124 27772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:37.774204 27772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:37.774235 27772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:37.774258 27772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:37.775656 27772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab
--principal=oryx/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:37.775856 27772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:37.776048 27772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:37.780895 27791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:37.781026 27772 server_base.cc:1034] running on GCE node
W20260809 16:11:37.780903 27784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:37.780927 27788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:37.781360 27772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:37.781599 27772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:37.782729 27772 hybrid_clock.cc:648] HybridClock initialized: now 1786291897782714 us; error 32 us; skew 500 ppm
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:37.785070 27772 init.cc:377] Logged in from keytab as oryx/127.16.224.130@KRBTEST.COM (short username oryx)
I20260809 16:11:37.785502 27772 webserver.cc:460] Webserver started at http://127.16.224.130:35343/ using document root <none> and password file <none>
I20260809 16:11:37.785701 27772 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:37.785749 27772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:37.785884 27772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:37.787070 27772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "299f27b9a8e14c34bd2174cabd1efa3d"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "b3b79a7d98a32bf994f219aa1d510985"
server_key_iv: "e322ba3a31c38391df28cfc1105e7658"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.787415 27772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "299f27b9a8e14c34bd2174cabd1efa3d"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "b3b79a7d98a32bf994f219aa1d510985"
server_key_iv: "e322ba3a31c38391df28cfc1105e7658"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.788718 27772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:37.789414 27804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.789577 27772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:37.789661 27772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "299f27b9a8e14c34bd2174cabd1efa3d"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "b3b79a7d98a32bf994f219aa1d510985"
server_key_iv: "e322ba3a31c38391df28cfc1105e7658"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.789721 27772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:37.805677 27772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:37.807554 27772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:37.807686 27772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:37.807896 27772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:37.808184 27772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:37.808218 27772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.808245 27772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:37.808262 27772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.814055 27772 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:40737
I20260809 16:11:37.814100 27974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:40737 every 8 connection(s)
I20260809 16:11:37.814424 27772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:37.816685 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 27772
I20260809 16:11:37.816790 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:37.816975 17282 external_mini_cluster.cc:1428] Setting key 999db057b28901d3bed83380377b23af
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:37.823503 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:37.815110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:41093 (local address 127.16.224.190:42645)
0809 16:11:37.815160 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:37.815163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:37.815353 (+ 190us) server_negotiation.cc:407] Connection header received
0809 16:11:37.815823 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:37.815825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:37.815835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:37.815884 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:37.816760 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.817379 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.818478 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.818733 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.818801 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:37.820776 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:37.820778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:37.820778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:37.820787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:37.822216 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.822611 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.822612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.822612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.822674 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.822885 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.822886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.822886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.823025 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:37.823096 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:37.823321 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 16:11:37.823406 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:37.823570 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:37.815083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:41093)
0809 16:11:37.815241 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:37.815275 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:37.815741 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:37.815939 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:37.815943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:37.816480 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:37.816663 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:37.816667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.817467 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.817468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:37.818384 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:37.818388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.818805 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.818806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:37.818909 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:37.819379 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:37.819387 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:37.820683 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:37.822297 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:37.822304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:37.822305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:37.822543 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:37.822738 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:37.822738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:37.822739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:37.822830 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:37.823082 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:37.823085 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:37.823195 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 16:11:37.823269 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:11:37.823406 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260809 16:11:37.824041 27975 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42645
I20260809 16:11:37.824132 27975 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:37.824302 27975 heartbeater.cc:511] Master 127.16.224.190:42645 requested a full tablet report, sending...
I20260809 16:11:37.824668 27441 ts_manager.cc:194] Registered new tserver with Master: 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130:40737)
I20260809 16:11:37.825160 27441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:41093
WARNING: no policy specified for oryx/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab.
Entry for principal oryx/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab.
I20260809 16:11:37.842706 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42645
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab
--principal=oryx/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:37.923207 27984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:37.923411 27984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:37.923434 27984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:37.924611 27984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:37.924655 27984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:37.924679 27984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:37.926081 27984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab
--principal=oryx/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:37.926271 27984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:37.926492 27984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:37.931134 27991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:37.931136 27996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:37.931154 27993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:37.931416 27984 server_base.cc:1034] running on GCE node
I20260809 16:11:37.931604 27984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:37.931833 27984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:37.932967 27984 hybrid_clock.cc:648] HybridClock initialized: now 1786291897932947 us; error 35 us; skew 500 ppm
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:37.935362 27984 init.cc:377] Logged in from keytab as oryx/127.16.224.131@KRBTEST.COM (short username oryx)
I20260809 16:11:37.935784 27984 webserver.cc:460] Webserver started at http://127.16.224.131:33455/ using document root <none> and password file <none>
I20260809 16:11:37.935993 27984 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:37.936040 27984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:37.936162 27984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:37.937130 27984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "592b82b92a0c4cc7af1edf8f657713d5"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "69150d35ee4734fbc00fc55924123213"
server_key_iv: "292f6ce5513197ac3e807836ff34a75e"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.937458 27984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "592b82b92a0c4cc7af1edf8f657713d5"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "69150d35ee4734fbc00fc55924123213"
server_key_iv: "292f6ce5513197ac3e807836ff34a75e"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.938798 27984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:37.939599 28004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.939786 27984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:37.939877 27984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "592b82b92a0c4cc7af1edf8f657713d5"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "69150d35ee4734fbc00fc55924123213"
server_key_iv: "292f6ce5513197ac3e807836ff34a75e"
server_key_version: "encryptionkey@0"
I20260809 16:11:37.939937 27984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:37.951225 27984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:37.952893 27984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:37.953032 27984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:37.953230 27984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:37.953521 27984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:37.953554 27984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.953583 27984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:37.953599 27984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:37.959244 27984 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:42243
I20260809 16:11:37.959283 28184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:42243 every 8 connection(s)
I20260809 16:11:37.959590 27984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:11:37 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:37.967540 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 27984
I20260809 16:11:37.967648 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:37.967854 17282 external_mini_cluster.cc:1428] Setting key 433f271fc46d1ed1ea25ef730e381839
I20260809 16:11:37.967870 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:37.960250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:57759 (local address 127.16.224.190:42645)
0809 16:11:37.960332 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:37.960334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:37.960497 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:11:37.960963 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:37.960964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:37.960975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:37.961019 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:37.961894 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.962469 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.963430 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.963620 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.963672 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:37.965616 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:37.965618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:37.965619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:37.965625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:37.966709 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.967035 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.967036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.967036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.967081 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:37.967249 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:37.967249 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:37.967250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:37.967342 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:37.967385 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:37.967734 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 16:11:37.967807 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:11:37.968091 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:37.960228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.131:57759)
0809 16:11:37.960388 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:37.960422 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:37.960890 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:37.961081 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:37.961084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:37.961639 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:37.961827 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:37.961832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.962541 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.962542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:37.963350 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:37.963353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:37.963678 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:37.963679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:37.963773 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:37.964215 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:37.964223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:37.965536 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:37.966769 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:37.966774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:37.966776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:37.966981 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:37.967124 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:37.967125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:37.967125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:37.967209 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:37.967393 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:37.967399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:37.967558 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 16:11:37.967706 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 16:11:37.967871 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260809 16:11:37.968600 28185 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42645
I20260809 16:11:37.968688 28185 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:37.968871 28185 heartbeater.cc:511] Master 127.16.224.190:42645 requested a full tablet report, sending...
I20260809 16:11:37.969218 27440 ts_manager.cc:194] Registered new tserver with Master: 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243)
I20260809 16:11:37.969588 27440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:57759
I20260809 16:11:37.979596 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.224.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.189.keytab.
Entry for principal oryx/127.16.224.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.189.keytab.
I20260809 16:11:38.002419 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:39427
--webserver_interface=127.16.224.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.189.keytab
--principal=oryx/127.16.224.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42645,127.16.224.189:39427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:38.082962 28191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:38.083158 28191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:38.083176 28191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:38.084263 28191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:38.084304 28191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:38.084316 28191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:38.084335 28191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:38.084348 28191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:38.085773 28191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42645,127.16.224.189:39427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.189.keytab
--principal=oryx/127.16.224.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:39427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.224.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
W20260809 16:11:38.085878 28191 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.16.224.190:42645,127.16.224.189:39427'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 16:11:38.086017 28191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:38.086217 28191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:38.090766 28208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.090768 28202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.090757 28197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.090930 28191 server_base.cc:1034] running on GCE node
I20260809 16:11:38.091135 28191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:38.091418 28191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:38.092558 28191 hybrid_clock.cc:648] HybridClock initialized: now 1786291898092547 us; error 29 us; skew 500 ppm
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:38.094919 28191 init.cc:377] Logged in from keytab as oryx/127.16.224.189@KRBTEST.COM (short username oryx)
I20260809 16:11:38.095353 28191 webserver.cc:460] Webserver started at http://127.16.224.189:40091/ using document root <none> and password file <none>
I20260809 16:11:38.095541 28191 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.095587 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.095710 28191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:38.096724 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/instance:
uuid: "6f296774bf534b358b473d832dea767f"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "ced77ae85788eb55df6534aaa693657c"
server_key_iv: "44165fd111cefc3de3cb776dd7201bd0"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.097062 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal/instance:
uuid: "6f296774bf534b358b473d832dea767f"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "ced77ae85788eb55df6534aaa693657c"
server_key_iv: "44165fd111cefc3de3cb776dd7201bd0"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.098342 28191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:11:38.099067 28219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.099222 28191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:38.099306 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
uuid: "6f296774bf534b358b473d832dea767f"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "ced77ae85788eb55df6534aaa693657c"
server_key_iv: "44165fd111cefc3de3cb776dd7201bd0"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.099371 28191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.111299 28191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:38.112932 28191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:38.113059 28191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:38.117059 28191 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.189:39427
I20260809 16:11:38.117094 28297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.189:39427 every 8 connection(s)
I20260809 16:11:38.117417 28191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
I20260809 16:11:38.118490 28298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:38.120038 28298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.189@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:38.127048 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 28191
I20260809 16:11:38.128386 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.120749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35372 (local address 127.16.224.190:42645)
0809 16:11:38.120807 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.120809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.121001 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:11:38.121451 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.121452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.121461 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.121518 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.122090 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.122620 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.123535 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.123712 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.123756 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.125629 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.125631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.125631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.125638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.126930 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.127284 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.127285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.127286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.127352 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.127622 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.127623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.127623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.127812 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.127878 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.128205 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.128299 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:11:38.128525 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.120719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35372)
0809 16:11:38.120890 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.120931 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.121373 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.121593 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.121596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.121843 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.122030 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.122035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.122679 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.122679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.123475 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.123477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.123763 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.123763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.123864 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.124277 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.124284 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.125545 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.126999 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.127002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.127004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.127228 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.127419 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.127421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.127421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.127556 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.127910 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.127916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.128070 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.128161 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.128310 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260809 16:11:38.129158 28298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.224.189@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.189@KRBTEST.COM for oryx/127.16.224.189@KRBTEST.COM
I20260809 16:11:38.135629 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.127570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.16.224.189:39427)
0809 16:11:38.127725 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.127730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.127841 (+ 111us) server_negotiation.cc:407] Connection header received
0809 16:11:38.127868 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.127870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.127884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.127953 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.128524 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.129300 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.130240 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.130610 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.130671 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.132987 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.132993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.132994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.132999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.134521 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.134872 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.134875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.134875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.134927 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.135102 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.135103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.135103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.135217 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.135342 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.135440 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.135520 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260809 16:11:38.136366 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.129369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.0.0.1:38558)
0809 16:11:38.129403 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.129413 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.129447 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.129636 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.129637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.129776 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.129823 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.129825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.130472 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.130473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.130972 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.130974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.131404 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.131405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.131454 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.131776 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.131780 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.133291 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.135387 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.135388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.135389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.135599 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.135827 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.135828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.135829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.135886 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.136110 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.136111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.136144 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.136230 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.136310 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 16:11:38.136473 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.129424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.16.224.189:39427)
0809 16:11:38.129548 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.129549 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.129553 (+ 4us) server_negotiation.cc:407] Connection header received
0809 16:11:38.129565 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.129565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.129571 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.129613 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.129888 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.130412 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.131044 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.131334 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.131406 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.133369 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.133370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.133370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.133376 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.135304 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.135685 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.135686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.135686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.135750 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.135934 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.135934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.135935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.136057 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.136119 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.136257 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.136372 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260809 16:11:38.137605 28298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Bootstrap starting.
I20260809 16:11:38.138197 28298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:38.138505 28298 log.cc:824] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:38.139256 28298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: No bootstrap required, opened a new log
I20260809 16:11:38.140743 28298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } }
I20260809 16:11:38.140885 28298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:38.140908 28298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f296774bf534b358b473d832dea767f, State: Initialized, Role: FOLLOWER
I20260809 16:11:38.141027 28298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } }
I20260809 16:11:38.141299 28309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } } }
I20260809 16:11:38.141396 28298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:38.141399 28309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:11:38.143435 28329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:11:38.143496 28329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:11:38.144965 28191 master_runner.cc:428] Detected that this master 6f296774bf534b358b473d832dea767f is joining an existing cluster
I20260809 16:11:38.145036 28191 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.224.189:39427
I20260809 16:11:38.152349 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.146217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.16.224.190:42645)
0809 16:11:38.146263 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.146266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.146425 (+ 159us) server_negotiation.cc:407] Connection header received
0809 16:11:38.146494 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.146495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.146506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.146554 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.146929 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.147728 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.148539 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.148734 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.148788 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.149801 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.149802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.149803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.149810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.151244 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.151629 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.151630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.151631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.151693 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.151870 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.151871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.151872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.151986 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.152040 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.152170 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.152243 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:11:38.152367 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.146175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35386)
0809 16:11:38.146359 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.146369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.146435 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.146601 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.146602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.146786 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.146854 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.146856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.147813 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.147814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.148460 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.148462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.148793 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.148794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.148854 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.149333 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.149337 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.149727 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.151329 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.151330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.151330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.151556 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.151746 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.151746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.151747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.151812 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.152039 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.152040 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.152083 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.152141 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.152268 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":49,"thread_start_us":82,"threads_started":1}
I20260809 16:11:38.153419 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.146235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38570 (local address 127.16.224.189:39427)
0809 16:11:38.146272 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.146274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.146526 (+ 252us) server_negotiation.cc:407] Connection header received
0809 16:11:38.146605 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.146606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.146613 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.146652 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.147026 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.147725 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.148877 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.149134 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.149201 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.150496 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.150497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.150497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.150503 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.152099 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.152649 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.152651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.152651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.152708 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.152910 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.152912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.152912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.153042 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.153101 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.153226 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.153338 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":19}
I20260809 16:11:38.153573 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.146303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.0.0.1:38570)
0809 16:11:38.146468 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.146478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.146539 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.146668 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.146668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.146879 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.146954 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.146956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.147803 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.147804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.148763 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.148766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.149207 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.149208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.149277 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.149982 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.149986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.150409 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.152182 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.152183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.152184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.152389 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.152772 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.152773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.152773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.152842 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.153103 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.153104 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.153142 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.153200 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.153485 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":24,"thread_start_us":104,"threads_started":1}
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.190@KRBTEST.COM for oryx/127.16.224.189@KRBTEST.COM
I20260809 16:11:38.161734 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.154859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.0.0.1:38580)
0809 16:11:38.154999 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.155043 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.155117 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.155242 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.155245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.155571 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.155644 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.155646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.156209 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.156210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.157164 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.157167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.157455 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.157456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.157539 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.157919 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.157923 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.159349 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.160811 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.160825 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.160827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.161012 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.161187 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.161187 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.161188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.161235 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.161454 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.161459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.161499 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.161558 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.161651 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260809 16:11:38.161755 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.154942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38580 (local address 127.16.224.189:39427)
0809 16:11:38.155027 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.155029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.155092 (+ 63us) server_negotiation.cc:407] Connection header received
0809 16:11:38.155188 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.155189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.155197 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.155237 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.155707 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.156147 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.157231 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.157396 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.157438 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.159427 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.159429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.159429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.159436 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.160730 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.161074 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.161075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.161076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.161130 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.161281 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.161282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.161282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.161404 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.161467 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.161593 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.161667 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 16:11:38.162398 27440 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.224.189:39427: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6f296774bf534b358b473d832dea767f"
member_type: NON_VOTER
last_known_addr {
host: "127.16.224.189"
port: 39427
}
attrs {
promote: true
}
}
dest_uuid: "d4cb60585dc14321b9637dcc9412ddf8"
cas_config_opid_index: -1
I20260809 16:11:38.162760 27440 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } }
I20260809 16:11:38.163226 28350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6f296774bf534b358b473d832dea767f (127.16.224.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } }
I20260809 16:11:38.163573 28261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:38.164238 28350 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.224.189:39427
I20260809 16:11:38.164337 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } }
I20260809 16:11:38.164403 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:38.164445 28261 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Refusing update from remote peer d4cb60585dc14321b9637dcc9412ddf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 16:11:38.164604 28191 master.cc:537] Master@127.16.224.189:39427 shutting down...
I20260809 16:11:38.164747 28348 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:11:38.165479 27419 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 16:11:38.165905 27419 consensus_peers.cc:487] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 -> Peer 6f296774bf534b358b473d832dea767f (127.16.224.189:39427): Couldn't send request to peer 6f296774bf534b358b473d832dea767f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:11:38.167449 28191 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:38.167537 28191 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:38.167587 28191 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: stopping tablet replica
I20260809 16:11:38.181277 28191 master.cc:552] Master@127.16.224.189:39427 shutdown complete.
I20260809 16:11:38.181342 28191 master_runner.cc:305] Clearing existing system tablet
I20260809 16:11:38.181478 28191 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.181528 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.182243 28191 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.002s
I20260809 16:11:38.182808 28358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.182934 28191 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.182988 28191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.183005 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
uuid: "6f296774bf534b358b473d832dea767f"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "ced77ae85788eb55df6534aaa693657c"
server_key_iv: "44165fd111cefc3de3cb776dd7201bd0"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.184293 28191 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:38.185438 28191 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:11:38.185495 28191 log.cc:1196] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 16:11:38.185658 28191 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Deleting consensus metadata
I20260809 16:11:38.185761 28191 master_runner.cc:315] Copying system tablet from 127.16.224.190:42645
I20260809 16:11:38.186388 28191 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: tablet copy: Beginning tablet copy session from remote peer at address 127.16.224.190:42645
I20260809 16:11:38.192477 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.186815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.16.224.190:42645)
0809 16:11:38.186875 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.186877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.186972 (+ 95us) server_negotiation.cc:407] Connection header received
0809 16:11:38.187041 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.187042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.187052 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.187099 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.187410 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.187920 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.188673 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.188881 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.188936 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.189885 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.189887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.189887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.189895 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.191291 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.191646 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.191647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.191647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.191707 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.191961 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.191962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.191962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.192083 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.192140 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.192284 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.192377 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:11:38.192490 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.186760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35400)
0809 16:11:38.186905 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.186915 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.186979 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.187101 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.187102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.187272 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.187337 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.187339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.187997 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.187999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.188593 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.188595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.188942 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.188943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.189006 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.189500 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.189505 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.189797 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.191370 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.191371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.191371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.191587 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.191775 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.191775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.191776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.191851 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.192141 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.192142 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.192190 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.192249 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.192377 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260809 16:11:38.193114 27480 tablet_copy_service.cc:140] P d4cb60585dc14321b9637dcc9412ddf8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6f296774bf534b358b473d832dea767f ({username='oryx', principal='oryx/127.16.224.189@KRBTEST.COM'} at 127.0.0.1:35400)
I20260809 16:11:38.193241 27480 tablet_copy_service.cc:161] P d4cb60585dc14321b9637dcc9412ddf8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6f296774bf534b358b473d832dea767f at {username='oryx', principal='oryx/127.16.224.189@KRBTEST.COM'} at 127.0.0.1:35400: session id = 6f296774bf534b358b473d832dea767f-00000000000000000000000000000000
I20260809 16:11:38.194007 27480 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:11:38.194689 28191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:11:38.195766 28191 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: tablet copy: Starting download of 0 data blocks...
I20260809 16:11:38.195910 28191 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: tablet copy: Starting download of 1 WAL segments...
I20260809 16:11:38.196871 28191 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:11:38.197682 27480 tablet_copy_service.cc:342] P d4cb60585dc14321b9637dcc9412ddf8: Request end of tablet copy session 6f296774bf534b358b473d832dea767f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.224.189@KRBTEST.COM'} at 127.0.0.1:35400
I20260809 16:11:38.197736 27480 tablet_copy_service.cc:433] P d4cb60585dc14321b9637dcc9412ddf8: ending tablet copy session 6f296774bf534b358b473d832dea767f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6f296774bf534b358b473d832dea767f
I20260809 16:11:38.200219 28191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:38.201272 28372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.201400 28368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.201537 28367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.202507 28191 server_base.cc:1034] running on GCE node
I20260809 16:11:38.202597 28191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:38.202736 28191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:38.203883 28191 hybrid_clock.cc:648] HybridClock initialized: now 1786291898203865 us; error 32 us; skew 500 ppm
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:38.205132 28191 init.cc:377] Logged in from keytab as oryx/127.16.224.189@KRBTEST.COM (short username oryx)
I20260809 16:11:38.205377 28191 webserver.cc:460] Webserver started at http://127.16.224.189:37605/ using document root <none> and password file <none>
I20260809 16:11:38.205506 28191 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.205541 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.206105 28191 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.206640 28384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.206789 28191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.206836 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
uuid: "6f296774bf534b358b473d832dea767f"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "ced77ae85788eb55df6534aaa693657c"
server_key_iv: "44165fd111cefc3de3cb776dd7201bd0"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.206894 28191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.214542 28191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:38.216075 28191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:38.219017 28191 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.189:39427
I20260809 16:11:38.219395 28470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.189:39427 every 8 connection(s)
I20260809 16:11:38.219429 28191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
I20260809 16:11:38.220470 28474 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:11:38.221078 28474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Bootstrap starting.
I20260809 16:11:38.224419 28474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:11:38.224666 28474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Bootstrap complete.
I20260809 16:11:38.224895 28474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } }
I20260809 16:11:38.224992 28474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f296774bf534b358b473d832dea767f, State: Initialized, Role: LEARNER
I20260809 16:11:38.225034 28474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } }
I20260809 16:11:38.225212 28474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:38.225226 28480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } }
I20260809 16:11:38.225355 28480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: LEARNER
I20260809 16:11:38.226851 28506 catalog_manager.cc:1259] Loaded cluster ID: 4743acfe20e34f9795156e1abdc3e193
I20260809 16:11:38.226897 28506 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: loading cluster ID for follower catalog manager: success
I20260809 16:11:38.227885 28506 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: acquiring CA information for follower catalog manager: success
I20260809 16:11:38.228137 28506 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:11:38.243330 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.237195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.16.224.189:39427)
0809 16:11:38.237341 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.237343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.237349 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:11:38.237371 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.237371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.237380 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.237434 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.237939 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.238617 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.239305 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.239505 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.239560 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.240679 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.240680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.240681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.240689 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.242062 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.242420 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.242422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.242422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.242499 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.242703 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.242704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.242704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.242879 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.242998 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.243128 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.243218 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.16.224.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.188.keytab.
Entry for principal oryx/127.16.224.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.188.keytab.
I20260809 16:11:38.267302 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:36765
--webserver_interface=127.16.224.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.188.keytab
--principal=oryx/127.16.224.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:38.354031 28510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:38.354247 28510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:38.354269 28510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:38.355489 28510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:38.355535 28510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:38.355547 28510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:38.355558 28510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:38.355569 28510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:38.357210 28510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.188.keytab
--principal=oryx/127.16.224.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:36765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.224.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:38.357491 28510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:38.357725 28510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:38.362571 28517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.362594 28516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.362674 28510 server_base.cc:1034] running on GCE node
W20260809 16:11:38.362731 28520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.362954 28510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:38.363245 28510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:38.364409 28510 hybrid_clock.cc:648] HybridClock initialized: now 1786291898364396 us; error 32 us; skew 500 ppm
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:38.366935 28510 init.cc:377] Logged in from keytab as oryx/127.16.224.188@KRBTEST.COM (short username oryx)
I20260809 16:11:38.367390 28510 webserver.cc:460] Webserver started at http://127.16.224.188:38061/ using document root <none> and password file <none>
I20260809 16:11:38.367586 28510 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.367633 28510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.367758 28510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:38.368798 28510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/instance:
uuid: "b9bc63a0403a4f2794d7c6248746385b"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "0130895e98e20e589e8d7c45db35c9c6"
server_key_iv: "afb27cafe7bba4930cb715d58b81288c"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.369135 28510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal/instance:
uuid: "b9bc63a0403a4f2794d7c6248746385b"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "0130895e98e20e589e8d7c45db35c9c6"
server_key_iv: "afb27cafe7bba4930cb715d58b81288c"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.370436 28510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:11:38.371129 28530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.371284 28510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:38.371378 28510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
uuid: "b9bc63a0403a4f2794d7c6248746385b"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "0130895e98e20e589e8d7c45db35c9c6"
server_key_iv: "afb27cafe7bba4930cb715d58b81288c"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.371436 28510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.384744 28510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:38.386881 28510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:38.387041 28510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:38.392160 28510 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.188:36765
I20260809 16:11:38.392251 28620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.188:36765 every 8 connection(s)
I20260809 16:11:38.392616 28510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
I20260809 16:11:38.393143 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 28510
I20260809 16:11:38.393793 28621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:38.396034 28621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.224.188@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.188@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:38.403645 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.393846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33964 (local address 127.16.224.188:36765)
0809 16:11:38.394034 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.394042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.394070 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:11:38.394595 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.394602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.394657 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.394787 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.395540 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.396566 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.397305 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.397841 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.397970 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.400250 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.400258 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.400259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.400271 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.401930 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.402351 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.402355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.402355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.402468 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.402678 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.402679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.402679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.402970 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.403089 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.403300 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.403435 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 16:11:38.404615 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.397344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.16.224.190:42645)
0809 16:11:38.397404 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.397406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.397413 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:38.397435 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.397435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.397444 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.397487 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.398078 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.398735 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.399566 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.399768 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.399831 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.402038 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.402040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.402040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.402049 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.403521 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.403905 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.403906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.403906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.403969 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.404148 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.404149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.404149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.404281 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.404348 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.404492 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.404554 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:11:38.404628 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.396635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35414)
0809 16:11:38.396809 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.396847 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.396901 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.397531 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.397534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.397900 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.397981 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.397983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.398851 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.398852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.399470 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.399472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.399868 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.399870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.399930 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.400386 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.400390 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.401968 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.403585 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.403591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.403593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.403831 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.404034 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.404034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.404035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.404092 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.404344 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.404349 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.404399 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.404474 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.404555 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260809 16:11:38.405086 28621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.188@KRBTEST.COM for oryx/127.16.224.189@KRBTEST.COM
I20260809 16:11:38.410409 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.405328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.0.0.1:38602)
0809 16:11:38.405369 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.405378 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.405417 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.405564 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.405565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.405703 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.405732 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.405733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.406289 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.406290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.406852 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.406854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.407180 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.407181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.407221 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.407507 (+ 286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.407511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.408524 (+ 1013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.409712 (+ 1188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.409713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.409713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.409853 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.409989 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.409989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.409990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.410029 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.410184 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.410185 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.410213 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.410256 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.410344 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:11:38.410439 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.405399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38602 (local address 127.16.224.189:39427)
0809 16:11:38.405467 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.405469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.405477 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:38.405501 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.405502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.405513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.405555 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.405782 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.406239 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.406918 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.407131 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.407179 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.408601 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.408603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.408603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.408610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.409651 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.409906 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.409907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.409907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.409948 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.410061 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.410061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.410062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.410147 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.410188 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.410273 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.410379 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:11:38.410794 28621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.188@KRBTEST.COM for oryx/127.16.224.188@KRBTEST.COM
I20260809 16:11:38.416083 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.411028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.0.0.1:33970)
0809 16:11:38.411068 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.411078 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.411115 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.411214 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.411215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.411342 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.411365 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.411366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.411963 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.411964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.412434 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.412435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.412754 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.412755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.412799 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.413074 (+ 275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.413077 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.414053 (+ 976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.415327 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.415328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.415328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.415500 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.415647 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.415647 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.415647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.415693 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.415855 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.415855 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.415887 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.415931 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.416006 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 16:11:38.416133 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.411065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.16.224.188:36765)
0809 16:11:38.411095 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.411097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.411104 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:38.411164 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.411165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.411173 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.411213 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.411424 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.411889 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.412499 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.412701 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.412752 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.414123 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.414124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.414124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.414130 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.415244 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.415563 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.415564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.415564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.415605 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.415733 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.415733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.415733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.415815 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.415856 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.415951 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.416048 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":18}
I20260809 16:11:38.417439 28621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Bootstrap starting.
I20260809 16:11:38.418231 28621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:38.418530 28621 log.cc:824] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:38.419199 28621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: No bootstrap required, opened a new log
I20260809 16:11:38.420563 28621 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } }
I20260809 16:11:38.420692 28621 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:38.420722 28621 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9bc63a0403a4f2794d7c6248746385b, State: Initialized, Role: FOLLOWER
I20260809 16:11:38.420818 28621 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } }
I20260809 16:11:38.421077 28621 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:38.421043 28626 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } } }
I20260809 16:11:38.421134 28626 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:11:38.422676 28645 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:11:38.422735 28645 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:11:38.424798 28510 master_runner.cc:428] Detected that this master b9bc63a0403a4f2794d7c6248746385b is joining an existing cluster
I20260809 16:11:38.424855 28510 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.224.188:36765
I20260809 16:11:38.433305 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.426205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.16.224.189:39427)
0809 16:11:38.426250 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.426252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.426491 (+ 239us) server_negotiation.cc:407] Connection header received
0809 16:11:38.426586 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.426587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.426598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.426627 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.427154 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.427931 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.428660 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.428876 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.428937 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.430146 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.430147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.430148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.430154 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.431516 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.431999 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.432000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.432001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.432061 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.432755 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.432756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.432757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.432866 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.432925 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.433121 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.433211 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:38.433658 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.426026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.0.0.1:38614)
0809 16:11:38.426433 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.426443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.426516 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.426691 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.426692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.426989 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.427082 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.427084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.428007 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.428008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.428581 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.428583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.428954 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.428955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.429023 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.429463 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.429467 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.429768 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.431762 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.431762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.431763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.431932 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.432121 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.432122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.432123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.432192 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.432941 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.432942 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.432987 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.433079 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.433180 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":46,"spinlock_wait_cycles":9600,"thread_start_us":110,"threads_started":1}
I20260809 16:11:38.434038 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.426453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.16.224.190:42645)
0809 16:11:38.426525 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.426528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.426536 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:38.426561 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.426562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.426573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.426621 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.428047 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.428796 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.429845 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.430048 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.430096 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.431340 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.431341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.431342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.431350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.432681 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.433062 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.433063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.433063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.433123 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.433312 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.433313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.433314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.433456 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.433501 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.433867 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.433951 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:11:38.435535 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.426022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35428)
0809 16:11:38.426173 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.426186 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.426244 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.426638 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.426639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.426986 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.427082 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.427084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.428905 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.428906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.429758 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.429761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.430111 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.430112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.430177 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.430821 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.430826 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.431211 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.432752 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.432753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.432753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.432979 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.433190 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.433191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.433191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.433249 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.433704 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.433705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.433750 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.435262 (+ 1512us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.435426 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":55,"thread_start_us":92,"threads_started":1}
I20260809 16:11:38.439284 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.426291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.0.0.1:33978)
0809 16:11:38.433874 (+ 7583us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.433884 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.433952 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.434102 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.434102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.434231 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.434287 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.434289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.435217 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.435217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.435919 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.435921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.436371 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.436372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.436441 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.436780 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.436784 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.437020 (+ 236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.438527 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.438528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.438528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.438670 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.438841 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.438841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.438842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.438881 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.439091 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.439092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.439121 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.439162 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.439228 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7572,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260809 16:11:38.439384 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.426536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.16.224.188:36765)
0809 16:11:38.426569 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.426570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.433931 (+ 7361us) server_negotiation.cc:407] Connection header received
0809 16:11:38.434016 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.434018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.434029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.434091 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.434435 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.435132 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.435989 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.436283 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.436363 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.437088 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.437089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.437089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.437095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.438459 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.438730 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.438731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.438732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.438786 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.438923 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.438924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.438924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.439038 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.439102 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.439186 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.439288 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":19}
I20260809 16:11:38.455083 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 27549
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291897, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.190@KRBTEST.COM for oryx/127.16.224.188@KRBTEST.COM
I20260809 16:11:38.462500 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.451746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.16.224.188:36765)
0809 16:11:38.451792 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.451794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.451802 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:38.451844 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.451845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.451856 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.451918 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.452190 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.452934 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.453581 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.453877 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.453942 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.456017 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.456018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.456019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.456027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.461277 (+ 5250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.461702 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.461703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.461703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.461766 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.461954 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.461955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.461955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.462084 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.462147 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.462291 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.462402 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:11:38.462517 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.451662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.0.0.1:33990)
0809 16:11:38.451725 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.451736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.451779 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.451924 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.451925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.452079 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.452129 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.452131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.453000 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.453001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.453518 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.453519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.453936 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.453937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.453980 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.454428 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.454432 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.455908 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.461372 (+ 5464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.461373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.461374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.461622 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.461830 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.461830 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.461831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.461898 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.462141 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.462142 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.462195 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.462261 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.462410 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:11:38.463146 27440 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.224.188:36765: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b"
member_type: NON_VOTER
last_known_addr {
host: "127.16.224.188"
port: 36765
}
attrs {
promote: true
}
}
dest_uuid: "d4cb60585dc14321b9637dcc9412ddf8"
cas_config_opid_index: 5
I20260809 16:11:38.463418 27440 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } }
I20260809 16:11:38.463784 28348 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b9bc63a0403a4f2794d7c6248746385b (127.16.224.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.464308 28576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:38.464854 28348 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.224.188:36765
I20260809 16:11:38.464932 28348 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.464989 28348 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:38.465240 28576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Refusing update from remote peer d4cb60585dc14321b9637dcc9412ddf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 16:11:38.465488 28510 master.cc:537] Master@127.16.224.188:36765 shutting down...
I20260809 16:11:38.465509 28350 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:38.466184 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:41065
--local_ip_for_outbound_sockets=127.16.224.129
--tserver_master_addrs=127.16.224.190:42645
--webserver_port=43035
--webserver_interface=127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab
--principal=oryx/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:38.466254 27422 consensus_peers.cc:487] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 -> Peer b9bc63a0403a4f2794d7c6248746385b (127.16.224.188:36765): Couldn't send request to peer b9bc63a0403a4f2794d7c6248746385b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:11:38.470026 28510 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:38.470124 28510 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:38.470170 28510 tablet_replica.cc:330] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: stopping tablet replica
I20260809 16:11:38.470497 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.463904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.0.0.1:38624)
0809 16:11:38.466869 (+ 2965us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.466879 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.466920 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.467121 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.467122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.467123 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:38.467165 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.467167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.468116 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.468117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.469332 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.469335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.470183 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.470184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.470251 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.470274 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.470386 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.470428 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2950}
I20260809 16:11:38.470727 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.464006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.16.224.189:39427)
0809 16:11:38.466716 (+ 2710us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.466718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.466968 (+ 250us) server_negotiation.cc:407] Connection header received
0809 16:11:38.466995 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.466996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.467015 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.467065 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:38.467233 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.468011 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.469431 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.470012 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.470079 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.470149 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.470624 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.470664 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2686}
I20260809 16:11:38.470943 28431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 LEARNER]: Refusing update from remote peer d4cb60585dc14321b9637dcc9412ddf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 16:11:38.471158 28350 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 16:11:38.471787 28431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b9bc63a0403a4f2794d7c6248746385b (127.16.224.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.472893 28480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4cb60585dc14321b9637dcc9412ddf8. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.472986 28480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: LEARNER
I20260809 16:11:38.472918 28663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: NON_VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: true } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.473140 28663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: LEARNER
I20260809 16:11:38.474121 28350 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: attempting to promote NON_VOTER 6f296774bf534b358b473d832dea767f to VOTER
I20260809 16:11:38.474431 28350 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } }
I20260809 16:11:38.474903 28431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 LEARNER]: Refusing update from remote peer d4cb60585dc14321b9637dcc9412ddf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 16:11:38.475071 28348 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 16:11:38.475127 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.474902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.0.0.1:34000)
0809 16:11:38.475053 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.475099 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:36765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 16:11:38.475551 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.475343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.0.0.1:34016)
0809 16:11:38.475500 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.475526 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:36765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260809 16:11:38.475617 27422 consensus_peers.cc:487] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 -> Peer b9bc63a0403a4f2794d7c6248746385b (127.16.224.188:36765): Couldn't send request to peer b9bc63a0403a4f2794d7c6248746385b. Status: Network error: Client connection negotiation failed: client connection to 127.16.224.188:36765: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 16:11:38.475946 28350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6f296774bf534b358b473d832dea767f (127.16.224.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.476112 28431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6f296774bf534b358b473d832dea767f (127.16.224.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.476774 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.476843 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:38.476889 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.476927 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:38.477151 28663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.477233 28663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:11:38.484871 28510 master.cc:552] Master@127.16.224.188:36765 shutdown complete.
I20260809 16:11:38.484929 28510 master_runner.cc:305] Clearing existing system tablet
I20260809 16:11:38.485051 28510 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.485095 28510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.485765 28510 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260809 16:11:38.486265 28666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.486433 28510 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.486488 28510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.486510 28510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
uuid: "b9bc63a0403a4f2794d7c6248746385b"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "0130895e98e20e589e8d7c45db35c9c6"
server_key_iv: "afb27cafe7bba4930cb715d58b81288c"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.487241 28510 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:38.488539 28510 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:11:38.488608 28510 log.cc:1196] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 16:11:38.488785 28510 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Deleting consensus metadata
I20260809 16:11:38.488891 28510 master_runner.cc:315] Copying system tablet from 127.16.224.190:42645
I20260809 16:11:38.489470 28510 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.224.190:42645
I20260809 16:11:38.509442 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.502390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.16.224.190:42645)
0809 16:11:38.502450 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.502453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.502544 (+ 91us) server_negotiation.cc:407] Connection header received
0809 16:11:38.502604 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.502605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.502617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.502692 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.503073 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.503831 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.504850 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.505147 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.505232 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.506490 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.506492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.506492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.506501 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.508189 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.508607 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.508608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.508608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.508668 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.508877 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.508877 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.508878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.509018 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.509081 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.509266 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.509343 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:11:38.509485 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.502286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35430)
0809 16:11:38.502469 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.502479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.502546 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.502707 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.502708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.502921 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.502979 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.502981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.503934 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.503936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.504757 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.504760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.505238 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.505239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.505302 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.505884 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.505889 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.506385 (+ 496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.508273 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.508275 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.508275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.508520 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.508731 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.508732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.508732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.508808 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.509087 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.509089 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.509157 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.509237 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.509368 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260809 16:11:38.510294 27480 tablet_copy_service.cc:140] P d4cb60585dc14321b9637dcc9412ddf8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b9bc63a0403a4f2794d7c6248746385b ({username='oryx', principal='oryx/127.16.224.188@KRBTEST.COM'} at 127.0.0.1:35430)
I20260809 16:11:38.510388 27480 tablet_copy_service.cc:161] P d4cb60585dc14321b9637dcc9412ddf8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b9bc63a0403a4f2794d7c6248746385b at {username='oryx', principal='oryx/127.16.224.188@KRBTEST.COM'} at 127.0.0.1:35430: session id = b9bc63a0403a4f2794d7c6248746385b-00000000000000000000000000000000
I20260809 16:11:38.510977 27480 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:11:38.511485 28510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:11:38.512869 28510 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: tablet copy: Starting download of 0 data blocks...
I20260809 16:11:38.513029 28510 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: tablet copy: Starting download of 1 WAL segments...
I20260809 16:11:38.513841 28510 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:11:38.514918 27480 tablet_copy_service.cc:342] P d4cb60585dc14321b9637dcc9412ddf8: Request end of tablet copy session b9bc63a0403a4f2794d7c6248746385b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.224.188@KRBTEST.COM'} at 127.0.0.1:35430
I20260809 16:11:38.514979 27480 tablet_copy_service.cc:433] P d4cb60585dc14321b9637dcc9412ddf8: ending tablet copy session b9bc63a0403a4f2794d7c6248746385b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b9bc63a0403a4f2794d7c6248746385b
I20260809 16:11:38.518174 28510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:38.519383 28684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.519395 28683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.519515 28686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.519886 28510 server_base.cc:1034] running on GCE node
I20260809 16:11:38.520020 28510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:38.520198 28510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:38.521392 28510 hybrid_clock.cc:648] HybridClock initialized: now 1786291898521384 us; error 25 us; skew 500 ppm
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:38.523027 28510 init.cc:377] Logged in from keytab as oryx/127.16.224.188@KRBTEST.COM (short username oryx)
I20260809 16:11:38.523372 28510 webserver.cc:460] Webserver started at http://127.16.224.188:44825/ using document root <none> and password file <none>
I20260809 16:11:38.523550 28510 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.523634 28510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.524413 28510 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:38.525069 28694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.525223 28510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.525326 28510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
uuid: "b9bc63a0403a4f2794d7c6248746385b"
format_stamp: "Formatted at 2026-08-09 16:11:38 on dist-test-slave-fgb3"
server_key: "0130895e98e20e589e8d7c45db35c9c6"
server_key_iv: "afb27cafe7bba4930cb715d58b81288c"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.525457 28510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.539197 28510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:38.541190 28510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:38.544811 28510 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.188:36765
I20260809 16:11:38.545233 28510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
I20260809 16:11:38.546489 28781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.188:36765 every 8 connection(s)
I20260809 16:11:38.546675 28782 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:11:38.547185 28782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Bootstrap starting.
I20260809 16:11:38.550418 28782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:11:38.550625 28782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: Bootstrap complete.
I20260809 16:11:38.550845 28782 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } }
I20260809 16:11:38.550942 28782 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b9bc63a0403a4f2794d7c6248746385b, State: Initialized, Role: LEARNER
I20260809 16:11:38.550980 28782 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } }
I20260809 16:11:38.551146 28782 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:38.553354 28801 catalog_manager.cc:1259] Loaded cluster ID: 4743acfe20e34f9795156e1abdc3e193
I20260809 16:11:38.553447 28801 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: loading cluster ID for follower catalog manager: success
I20260809 16:11:38.554409 28786 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.554517 28786 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: This master's current role is: LEARNER
I20260809 16:11:38.554765 28801 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: acquiring CA information for follower catalog manager: success
I20260809 16:11:38.555117 28801 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 16:11:38.566526 28660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:38.566716 28660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:38.566740 28660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:38.567874 28660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:38.567914 28660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:38.567939 28660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:38.569456 28660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab
--principal=oryx/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:41065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=43035
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:38.569682 28660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:38.569903 28660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:38.575718 28807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.575742 28813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.575754 28811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.576016 28660 server_base.cc:1034] running on GCE node
I20260809 16:11:38.576208 28660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:38.576445 28660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:38.577596 28660 hybrid_clock.cc:648] HybridClock initialized: now 1786291898577583 us; error 29 us; skew 500 ppm
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:38.580121 28660 init.cc:377] Logged in from keytab as oryx/127.16.224.129@KRBTEST.COM (short username oryx)
I20260809 16:11:38.580669 28660 webserver.cc:460] Webserver started at http://127.16.224.129:43035/ using document root <none> and password file <none>
I20260809 16:11:38.580881 28660 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.580935 28660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.582182 28660 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:38.582728 28824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.582866 28660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.582952 28660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "ebba9f24fa6a45d89db719d5530fb661"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "5ff205bb9853f30eb5c995aebe04384c"
server_key_iv: "a227d894d1c42241712290ef4d44b77c"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.583307 28660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.599432 28660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:38.601341 28660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:38.601482 28660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:38.601706 28660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:38.602031 28660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:38.602066 28660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.602097 28660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:38.602119 28660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.608237 28660 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:41065
I20260809 16:11:38.608311 28994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:41065 every 8 connection(s)
I20260809 16:11:38.608615 28660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:38.614140 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 28660
I20260809 16:11:38.614355 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 27772
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.189@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.188@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:38.622467 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.609610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:59027 (local address 127.16.224.189:39427)
0809 16:11:38.609661 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.609663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.609675 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:11:38.610258 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.610261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.610274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.610303 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.610976 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.611492 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.612883 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.613175 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.613239 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.615737 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.615739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.615739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.615747 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.619852 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.621308 (+ 1456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.621309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.621310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.621365 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.621588 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.621589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.621589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.621731 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.621781 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.622242 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.622360 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:38.623100 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.609359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.16.224.129:59027)
0809 16:11:38.609541 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.609592 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.610127 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.610413 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.610417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.610674 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.610880 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.610885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.611587 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.611588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.612804 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.612807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.613258 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.613259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.613415 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.613984 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.613992 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.615642 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.620314 (+ 4672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.620324 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.620326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.621239 (+ 913us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.621420 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.621421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.621421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.621521 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.621825 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.621832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.622042 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.622581 (+ 539us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.622825 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260809 16:11:38.625207 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.609381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:45643 (local address 127.16.224.190:42645)
0809 16:11:38.609432 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.609434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.609835 (+ 401us) server_negotiation.cc:407] Connection header received
0809 16:11:38.610221 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.610222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.610235 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.610301 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.611579 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.612049 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.612922 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.613219 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.613273 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.618153 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.618154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.618155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.618162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.620990 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.624392 (+ 3402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.624393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.624393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.624455 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.624680 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.624681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.624681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.624823 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.624871 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.625036 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.625111 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:11:38.625255 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.609650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.129:45643)
0809 16:11:38.609758 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.609775 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.610135 (+ 360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.610675 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.610676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.610829 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.610891 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.610893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.612127 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.612128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.612807 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.612809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.613333 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.613334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.613413 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.614014 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.614020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.618063 (+ 4043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.624054 (+ 5991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.624055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.624056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.624295 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.624534 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.624535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.624536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.624614 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.624870 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.624871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.624930 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.625011 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.625130 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
I20260809 16:11:38.628015 28997 heartbeater.cc:347] Connected to a master server at 127.16.224.189:39427
I20260809 16:11:38.628103 28997 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.628264 28996 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42645
I20260809 16:11:38.628291 28997 heartbeater.cc:511] Master 127.16.224.189:39427 requested a full tablet report, sending...
I20260809 16:11:38.628317 28996 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.628386 28996 heartbeater.cc:511] Master 127.16.224.190:42645 requested a full tablet report, sending...
I20260809 16:11:38.628681 27440 ts_manager.cc:194] Re-registered known tserver with Master: ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129:41065)
I20260809 16:11:38.628835 28409 ts_manager.cc:194] Registered new tserver with Master: ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129:41065)
I20260809 16:11:38.629179 27440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:45643
I20260809 16:11:38.631314 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.609389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:45977 (local address 127.16.224.188:36765)
0809 16:11:38.611007 (+ 1618us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.611009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.611014 (+ 5us) server_negotiation.cc:407] Connection header received
0809 16:11:38.611036 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.611036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.611045 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.611099 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.611496 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.612515 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.613296 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.613585 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.613646 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.617787 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.617790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.617790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.617799 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.630248 (+ 12449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.630594 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.630595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.630595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.630660 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.630821 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.630822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.630822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.630940 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.630996 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.631119 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.631210 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1589,"thread_start_us":59,"threads_started":1}
I20260809 16:11:38.631500 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.609359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.16.224.129:45977)
0809 16:11:38.609542 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.609592 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.610136 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.611132 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.611133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.611340 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.611387 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.611389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.612605 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.612606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.613202 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.613204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.613697 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.613698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.613746 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.614140 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.614144 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.617695 (+ 3551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.630353 (+ 12658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.630354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.630354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.630524 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.630714 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.630715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.630715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.630763 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.630994 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.630995 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.631036 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.631089 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.631207 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":15,"spinlock_wait_cycles":8960,"thread_start_us":101,"threads_started":1}
I20260809 16:11:38.631913 28995 heartbeater.cc:347] Connected to a master server at 127.16.224.188:36765
I20260809 16:11:38.631969 28995 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.632035 28995 heartbeater.cc:511] Master 127.16.224.188:36765 requested a full tablet report, sending...
I20260809 16:11:38.632416 28718 ts_manager.cc:194] Registered new tserver with Master: ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129:41065)
I20260809 16:11:38.633064 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:40737
--local_ip_for_outbound_sockets=127.16.224.130
--tserver_master_addrs=127.16.224.190:42645
--webserver_port=35343
--webserver_interface=127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab
--principal=oryx/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:38.716930 29005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:38.717139 29005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:38.717161 29005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:38.718379 29005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:38.718425 29005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:38.718456 29005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:38.719933 29005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab
--principal=oryx/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:40737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=35343
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:38.720166 29005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:38.720386 29005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:38.725248 29017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.725282 29020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.725248 29018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.725296 29005 server_base.cc:1034] running on GCE node
I20260809 16:11:38.725695 29005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:38.725936 29005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:38.727077 29005 hybrid_clock.cc:648] HybridClock initialized: now 1786291898727064 us; error 31 us; skew 500 ppm
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:38.729655 29005 init.cc:377] Logged in from keytab as oryx/127.16.224.130@KRBTEST.COM (short username oryx)
I20260809 16:11:38.730084 29005 webserver.cc:460] Webserver started at http://127.16.224.130:35343/ using document root <none> and password file <none>
I20260809 16:11:38.730357 29005 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.730438 29005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.731832 29005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:38.732383 29029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.732530 29005 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:38.732616 29005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "299f27b9a8e14c34bd2174cabd1efa3d"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "b3b79a7d98a32bf994f219aa1d510985"
server_key_iv: "e322ba3a31c38391df28cfc1105e7658"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.732965 29005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.748039 29005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:38.749984 29005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:38.750123 29005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:38.750376 29005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:38.750694 29005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:38.750728 29005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.750756 29005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:38.750774 29005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.756488 29005 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:40737
I20260809 16:11:38.756560 29199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:40737 every 8 connection(s)
I20260809 16:11:38.756879 29005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:38.758368 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 29005
I20260809 16:11:38.758531 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 27984
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.188@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.189@KRBTEST.COM
I20260809 16:11:38.772426 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.757891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:52499 (local address 127.16.224.189:39427)
0809 16:11:38.757939 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.757941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.757996 (+ 55us) server_negotiation.cc:407] Connection header received
0809 16:11:38.761997 (+ 4001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.761999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.762014 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.762045 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.763042 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.763981 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765432 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.765801 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765851 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.769454 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.769456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.769456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.769464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.771014 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.771482 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.771484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.771484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.771550 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.771759 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.771760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.771761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.771896 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.771945 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.772233 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.772321 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:11:38.772423 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.757693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:42235 (local address 127.16.224.188:36765)
0809 16:11:38.757800 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.757802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.758009 (+ 207us) server_negotiation.cc:407] Connection header received
0809 16:11:38.761968 (+ 3959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.761970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.761986 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.762012 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.763025 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.763783 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765417 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.765732 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765766 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.769088 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.769090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.769090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.769099 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.770688 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.771175 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.771177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.771177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.771236 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.771612 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.771613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.771613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.771774 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.771827 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.772233 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.772321 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 16:11:38.772979 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.757575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.16.224.130:42235)
0809 16:11:38.757850 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.757896 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.761851 (+ 3955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.762189 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.762193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.762627 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.762922 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.762927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.763960 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.763961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.765314 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.765318 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.765858 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.766015 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.766712 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.766722 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.768974 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.770784 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.770794 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.770797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.771093 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.771302 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.771302 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.771303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.771522 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.771859 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.771879 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.772056 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.772521 (+ 465us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.772729 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":6,"thread_start_us":107,"threads_started":1}
I20260809 16:11:38.773370 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.757575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.16.224.130:52499)
0809 16:11:38.757889 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.757897 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.761843 (+ 3946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.762189 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.762195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.762640 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.762923 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.762931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.764088 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.764089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.765316 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.765320 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.765942 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.766025 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.767014 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.767020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.769375 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.771099 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.771100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.771101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.771289 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.771621 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.771621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.771622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.771683 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.771976 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.771977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.772056 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.772897 (+ 841us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.772988 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":66,"thread_start_us":85,"threads_started":1}
I20260809 16:11:38.774051 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.757750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:57171 (local address 127.16.224.190:42645)
0809 16:11:38.757841 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.757842 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.758093 (+ 251us) server_negotiation.cc:407] Connection header received
0809 16:11:38.763021 (+ 4928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.763023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.763036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.763067 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.763436 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.764116 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765417 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.765717 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765758 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.769179 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.769181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.769181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.769191 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.772198 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.773018 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.773019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.773020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.773092 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.773455 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.773457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.773457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.773624 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.773676 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.773916 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.773989 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 16:11:38.774176 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.757575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:57171)
0809 16:11:38.758013 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.758024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.762906 (+ 4882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.763130 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.763131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.763285 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.763348 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.763349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.764201 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.764202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.765321 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.765322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.765856 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.765857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.766015 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.766855 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.766859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.769078 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.772639 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.772640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.772641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.772910 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.773180 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.773181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.773181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.773359 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.773731 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.773732 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.773785 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.773875 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.773989 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":114,"thread_start_us":80,"threads_started":1}
I20260809 16:11:38.774601 29205 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42645
I20260809 16:11:38.774704 29205 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.774894 29205 heartbeater.cc:511] Master 127.16.224.190:42645 requested a full tablet report, sending...
I20260809 16:11:38.775313 27440 ts_manager.cc:194] Re-registered known tserver with Master: 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130:40737)
I20260809 16:11:38.775686 27440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:57171
I20260809 16:11:38.787377 29206 heartbeater.cc:347] Connected to a master server at 127.16.224.189:39427
I20260809 16:11:38.787377 29202 heartbeater.cc:347] Connected to a master server at 127.16.224.188:36765
I20260809 16:11:38.787496 29202 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.787496 29206 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.787637 29202 heartbeater.cc:511] Master 127.16.224.188:36765 requested a full tablet report, sending...
I20260809 16:11:38.787637 29206 heartbeater.cc:511] Master 127.16.224.189:39427 requested a full tablet report, sending...
I20260809 16:11:38.788036 28718 ts_manager.cc:194] Registered new tserver with Master: 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130:40737)
I20260809 16:11:38.788036 28409 ts_manager.cc:194] Registered new tserver with Master: 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130:40737)
I20260809 16:11:38.791169 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:42243
--local_ip_for_outbound_sockets=127.16.224.131
--tserver_master_addrs=127.16.224.190:42645
--webserver_port=33455
--webserver_interface=127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab
--principal=oryx/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:38.873318 29211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:38.873518 29211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:38.873549 29211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:38.874717 29211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:38.874758 29211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:38.874792 29211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:38.876340 29211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab
--principal=oryx/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:42243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=33455
--tserver_master_addrs=127.16.224.190:42645,127.16.224.189:39427,127.16.224.188:36765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:38.876598 29211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:38.876816 29211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:38.881688 29224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.881769 29211 server_base.cc:1034] running on GCE node
W20260809 16:11:38.881698 29225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:38.881752 29228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:38.882103 29211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:38.882357 29211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:38.883503 29211 hybrid_clock.cc:648] HybridClock initialized: now 1786291898883490 us; error 30 us; skew 500 ppm
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:38.886049 29211 init.cc:377] Logged in from keytab as oryx/127.16.224.131@KRBTEST.COM (short username oryx)
I20260809 16:11:38.886612 29211 webserver.cc:460] Webserver started at http://127.16.224.131:33455/ using document root <none> and password file <none>
I20260809 16:11:38.886842 29211 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:38.886904 29211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:38.888273 29211 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:38.888825 29236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.888996 29211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.889084 29211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "592b82b92a0c4cc7af1edf8f657713d5"
format_stamp: "Formatted at 2026-08-09 16:11:37 on dist-test-slave-fgb3"
server_key: "69150d35ee4734fbc00fc55924123213"
server_key_iv: "292f6ce5513197ac3e807836ff34a75e"
server_key_version: "encryptionkey@0"
I20260809 16:11:38.889433 29211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:38.899075 29211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:38.900905 29211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:38.901016 29211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:38.901194 29211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:38.901449 29211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:38.901479 29211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.901500 29211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:38.901516 29211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:38.907344 29211 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:42243
I20260809 16:11:38.907405 29418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:42243 every 8 connection(s)
I20260809 16:11:38.907737 29211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.188@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
Aug 09 16:11:38 dist-test-slave-fgb3 krb5kdc[27366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291898, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.189@KRBTEST.COM
I20260809 16:11:38.916345 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 29211
I20260809 16:11:38.918799 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.908496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.16.224.131:51779)
0809 16:11:38.908668 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.908705 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.909987 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.910242 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.910245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.910581 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.910780 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.910786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.911708 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.911709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.912650 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.912653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.913372 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.913373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.913455 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.913779 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.913783 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.915263 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.917087 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.917094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.917096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.917386 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.917640 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.917641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.917641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.917769 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.918102 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.918108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.918268 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.918403 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.918561 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260809 16:11:38.919144 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.908466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:51779 (local address 127.16.224.188:36765)
0809 16:11:38.908525 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.908527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.908804 (+ 277us) server_negotiation.cc:407] Connection header received
0809 16:11:38.910101 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.910102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.910113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.910143 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.910872 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.911628 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.913114 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.913300 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.913354 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.915363 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.915365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.915365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.915372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.916995 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.917490 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.917492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.917493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.917556 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.917861 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.917862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.917862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.918014 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.918071 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.918954 (+ 883us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.919045 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:11:38.919445 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.908950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.16.224.131:58337)
0809 16:11:38.909215 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.909222 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.909993 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.910296 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.910297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.910622 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.910781 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.910788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.911879 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.911881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.912527 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.912529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.912932 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.912933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.913089 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.913996 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.914000 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.916333 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.918181 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.918182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.918183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.918419 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.918616 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.918617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.918617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.918670 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.919150 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.919151 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.919195 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.919256 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.919358 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":5,"thread_start_us":196,"threads_started":1}
I20260809 16:11:38.919638 29419 heartbeater.cc:347] Connected to a master server at 127.16.224.188:36765
I20260809 16:11:38.919731 29419 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.919919 29419 heartbeater.cc:511] Master 127.16.224.188:36765 requested a full tablet report, sending...
I20260809 16:11:38.920157 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.908998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:58337 (local address 127.16.224.189:39427)
0809 16:11:38.909050 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.909052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.909271 (+ 219us) server_negotiation.cc:407] Connection header received
0809 16:11:38.910221 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.910223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.910234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.910285 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.910900 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.911780 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.912608 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.912861 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.912923 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.916439 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.916441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.916442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.916449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.917969 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.918489 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.918490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.918490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.918551 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.918729 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.918730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.918730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.918849 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.918908 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.919990 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.920066 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:11:38.920379 28718 ts_manager.cc:194] Registered new tserver with Master: 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243)
I20260809 16:11:38.920449 29424 heartbeater.cc:347] Connected to a master server at 127.16.224.189:39427
I20260809 16:11:38.920499 29424 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.920571 29424 heartbeater.cc:511] Master 127.16.224.189:39427 requested a full tablet report, sending...
I20260809 16:11:38.921641 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.908817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:47223 (local address 127.16.224.190:42645)
0809 16:11:38.908866 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.908868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.909167 (+ 299us) server_negotiation.cc:407] Connection header received
0809 16:11:38.910096 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.910098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.910112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.910165 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.910898 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.911389 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.912735 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.913016 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.913064 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.915868 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.915869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.915870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.915878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.917934 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.920907 (+ 2973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.920908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.920909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.920965 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.921135 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.921136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.921136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.921253 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.921310 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.921482 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.921559 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:11:38.921727 28409 ts_manager.cc:194] Registered new tserver with Master: 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243)
I20260809 16:11:38.922156 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.908496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.131:47223)
0809 16:11:38.909100 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.909109 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.909989 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.910240 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.910244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.910620 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:38.910781 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.910788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.911486 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.911487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.912652 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.912653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.913082 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.913082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.913137 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.913656 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:38.913667 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:38.915735 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:38.920663 (+ 4928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.920664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.920664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.920835 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.921020 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:38.921021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:38.921021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:38.921077 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:38.921338 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:38.921339 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:38.921379 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.921762 (+ 383us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.922070 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":52,"thread_start_us":94,"threads_started":1}
I20260809 16:11:38.922825 29420 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42645
I20260809 16:11:38.922880 29420 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:38.922941 29420 heartbeater.cc:511] Master 127.16.224.190:42645 requested a full tablet report, sending...
I20260809 16:11:38.923157 27440 ts_manager.cc:194] Re-registered known tserver with Master: 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243)
I20260809 16:11:38.923615 27440 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:47223
I20260809 16:11:38.926414 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.917663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.16.224.190:42645)
0809 16:11:38.917972 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.917974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.918434 (+ 460us) server_negotiation.cc:407] Connection header received
0809 16:11:38.918526 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.918527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.918539 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.918582 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.919290 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.920015 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.921240 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.921527 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.921580 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.923304 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.923306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.923306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.923324 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.925160 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.925613 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.925614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.925615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.925672 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.925866 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.925867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.925867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.925992 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.926085 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.926198 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.926293 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":98,"threads_started":1}
I20260809 16:11:38.926863 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.918288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.16.224.189:39427)
0809 16:11:38.918405 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.918407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.918413 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:11:38.918434 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.918435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.918447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.919625 (+ 1178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.919667 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.920477 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.921949 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.922221 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.922629 (+ 408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.923670 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.923671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.923672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.923679 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.925287 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.925855 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.925856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.925856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.925927 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.926093 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.926094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.926094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.926214 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.926652 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.926692 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.926771 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
I20260809 16:11:38.929068 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.918184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34020 (local address 127.16.224.188:36765)
0809 16:11:38.919261 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.919263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.921323 (+ 2060us) server_negotiation.cc:407] Connection header received
0809 16:11:38.921415 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.921416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.921425 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.921450 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:38.922442 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.923583 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.924862 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.925096 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.925148 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.926518 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:38.926520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:38.926520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:38.926528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:38.928132 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.928419 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.928420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.928421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.928470 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:38.928630 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:38.928631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:38.928632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:38.928736 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:38.928812 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.928920 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.928984 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1062,"thread_start_us":60,"threads_started":1}
I20260809 16:11:38.928164 27440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:38.929795 27440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:38.937372 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.933662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57684 (local address 127.16.224.131:42243)
0809 16:11:38.933825 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.933829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.933837 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:38.933865 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.933867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.933879 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.933963 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:38.934117 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.935172 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.936482 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.937036 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.937114 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.937173 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.937259 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.937293 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260809 16:11:38.937377 29457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.933365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:42243 (local address 127.0.0.1:57684)
0809 16:11:38.933593 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.933602 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.933641 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.933972 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.933973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.933975 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:38.934037 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.934039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.935264 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.935265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.936396 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.936399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.937117 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.937118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.937185 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.937190 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.937244 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.937284 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":32,"thread_start_us":180,"threads_started":1}
I20260809 16:11:38.937378 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.933275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40737 (local address 127.0.0.1:43624)
0809 16:11:38.933307 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.933325 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.933359 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.933922 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.933923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.933925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:38.933966 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.933967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.935128 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.935128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.936281 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.936284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.937165 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.937166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.937228 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.937232 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.937261 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.937300 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":21}
I20260809 16:11:38.937589 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.933549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.16.224.129:41065)
0809 16:11:38.934024 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.934029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.934037 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:38.934074 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.934077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.934091 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.934159 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:38.934365 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.935353 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.936287 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.937065 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.937155 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.937215 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.937457 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.937502 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":78,"threads_started":1}
I20260809 16:11:38.937721 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.933307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:41065 (local address 127.0.0.1:53508)
0809 16:11:38.933539 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.933549 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.933601 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.934224 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.934224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.934226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:38.934259 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.934260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.935437 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.935437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.936211 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.936214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.937247 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.937247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.937302 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.937305 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.937361 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.937394 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260809 16:11:38.938426 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.933405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.16.224.130:40737)
0809 16:11:38.933753 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.933757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.933768 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:38.933813 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.933816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.933832 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.933909 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:38.934048 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.935046 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.936365 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.937072 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.938169 (+ 1097us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.938232 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.938258 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.938301 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":98,"threads_started":1}
I20260809 16:11:38.938576 29306 tablet_service.cc:1467] Processing CreateTablet for tablet c284dbf63bcc40e7bab2458508cdb0fc (DEFAULT_TABLE table=test-table [id=b93b44acec114aefa8493176eeb1d508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:38.938578 28895 tablet_service.cc:1467] Processing CreateTablet for tablet c284dbf63bcc40e7bab2458508cdb0fc (DEFAULT_TABLE table=test-table [id=b93b44acec114aefa8493176eeb1d508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:38.938977 29306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c284dbf63bcc40e7bab2458508cdb0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:38.938980 28895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c284dbf63bcc40e7bab2458508cdb0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:38.938992 29101 tablet_service.cc:1467] Processing CreateTablet for tablet c284dbf63bcc40e7bab2458508cdb0fc (DEFAULT_TABLE table=test-table [id=b93b44acec114aefa8493176eeb1d508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:38.939283 29101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c284dbf63bcc40e7bab2458508cdb0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:38.942240 29461 tablet_bootstrap.cc:492] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Bootstrap starting.
I20260809 16:11:38.942629 29462 tablet_bootstrap.cc:492] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Bootstrap starting.
I20260809 16:11:38.942965 29461 tablet_bootstrap.cc:654] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:38.942961 29463 tablet_bootstrap.cc:492] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Bootstrap starting.
I20260809 16:11:38.943251 29461 log.cc:824] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:38.943305 29462 tablet_bootstrap.cc:654] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:38.943562 29463 tablet_bootstrap.cc:654] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:38.943602 29462 log.cc:824] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:38.943988 29463 log.cc:824] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:38.944011 29461 tablet_bootstrap.cc:492] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: No bootstrap required, opened a new log
I20260809 16:11:38.944073 29461 ts_tablet_manager.cc:1397] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:38.944403 29462 tablet_bootstrap.cc:492] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: No bootstrap required, opened a new log
I20260809 16:11:38.944475 29462 ts_tablet_manager.cc:1397] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:38.944707 29463 tablet_bootstrap.cc:492] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: No bootstrap required, opened a new log
I20260809 16:11:38.944777 29463 ts_tablet_manager.cc:1397] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:38.946130 29461 raft_consensus.cc:348] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:38.946269 29461 raft_consensus.cc:374] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:38.946300 29461 raft_consensus.cc:729] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebba9f24fa6a45d89db719d5530fb661, State: Initialized, Role: FOLLOWER
I20260809 16:11:38.946470 29461 consensus_queue.cc:260] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:38.946557 29462 raft_consensus.cc:348] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:38.946712 29461 ts_tablet_manager.cc:1428] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:11:38.946719 29462 raft_consensus.cc:374] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:38.946753 29462 raft_consensus.cc:729] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299f27b9a8e14c34bd2174cabd1efa3d, State: Initialized, Role: FOLLOWER
I20260809 16:11:38.946870 29462 consensus_queue.cc:260] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:38.946906 29463 raft_consensus.cc:348] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:38.947046 29463 raft_consensus.cc:374] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:38.947091 29463 raft_consensus.cc:729] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 592b82b92a0c4cc7af1edf8f657713d5, State: Initialized, Role: FOLLOWER
I20260809 16:11:38.947135 29462 ts_tablet_manager.cc:1428] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:11:38.947199 29463 consensus_queue.cc:260] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:38.947470 29463 ts_tablet_manager.cc:1428] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:11:38.947604 28996 heartbeater.cc:503] Master 127.16.224.190:42645 was elected leader, sending a full tablet report...
I20260809 16:11:38.947732 29420 heartbeater.cc:503] Master 127.16.224.190:42645 was elected leader, sending a full tablet report...
I20260809 16:11:38.948753 29205 heartbeater.cc:503] Master 127.16.224.190:42645 was elected leader, sending a full tablet report...
I20260809 16:11:38.963304 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.960998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.16.224.188:36765)
0809 16:11:38.961055 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:38.961057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:38.961089 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:11:38.961115 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:38.961116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:38.961142 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:38.961212 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:38.961325 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.961806 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.962704 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.963084 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.963141 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.963205 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:38.963218 (+ 13us) server_negotiation.cc:299] Negotiation successful
0809 16:11:38.963242 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:11:38.963304 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:38.960943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.0.0.1:34036)
0809 16:11:38.960993 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:38.961005 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:38.961051 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:38.961226 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:38.961227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:38.961229 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:38.961275 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.961276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.961855 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.961855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.962631 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:38.962634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:38.963142 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:38.963143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:38.963180 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:38.963183 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:38.963208 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 16:11:38.963244 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:11:38.963667 28739 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260809 16:11:38.964548 28786 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4cb60585dc14321b9637dcc9412ddf8. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:38.964650 28786 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: This master's current role is: LEARNER
W20260809 16:11:39.008635 29207 tablet.cc:2369] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:39.101258 29468 raft_consensus.cc:482] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:39.101418 29468 raft_consensus.cc:504] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:39.101776 29468 leader_election.cc:288] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130:40737), 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243)
I20260809 16:11:39.104784 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.101962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40737 (local address 127.16.224.129:53207)
0809 16:11:39.102022 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:39.102033 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:39.102098 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:39.102264 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:39.102265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:39.102267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:39.102382 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:39.102384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.103112 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.103113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:39.103919 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:39.103922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.104529 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.104530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:39.104605 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.104609 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:39.104671 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:11:39.104717 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:11:39.104776 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.102043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:53207 (local address 127.16.224.130:40737)
0809 16:11:39.102117 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:39.102119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:39.102146 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:11:39.102179 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:39.102180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:39.102189 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:39.102234 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:39.102447 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.103046 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.104011 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.104445 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.104496 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.104504 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:39.104681 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 16:11:39.104707 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:11:39.104992 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.102071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:42243 (local address 127.16.224.129:54775)
0809 16:11:39.102113 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:39.102122 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:39.102162 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:39.102348 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:39.102349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:39.102350 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:39.102692 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:39.102695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.103377 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.103377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:39.104137 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:39.104139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.104737 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.104738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:39.104824 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.104828 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:39.104880 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:11:39.104914 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:11:39.105037 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.102152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:54775 (local address 127.16.224.131:42243)
0809 16:11:39.102199 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:39.102200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:39.102209 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:39.102234 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:39.102235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:39.102260 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:39.102335 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:39.102769 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.103304 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.104243 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.104668 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.104727 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.104735 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:39.104936 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:11:39.104979 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:11:39.105201 29130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c284dbf63bcc40e7bab2458508cdb0fc" candidate_uuid: "ebba9f24fa6a45d89db719d5530fb661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" is_pre_election: true
I20260809 16:11:39.105369 29130 raft_consensus.cc:2454] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebba9f24fa6a45d89db719d5530fb661 in term 0.
I20260809 16:11:39.105620 28831 leader_election.cc:302] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 299f27b9a8e14c34bd2174cabd1efa3d, ebba9f24fa6a45d89db719d5530fb661; no voters:
I20260809 16:11:39.105641 29345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c284dbf63bcc40e7bab2458508cdb0fc" candidate_uuid: "ebba9f24fa6a45d89db719d5530fb661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592b82b92a0c4cc7af1edf8f657713d5" is_pre_election: true
I20260809 16:11:39.105742 29468 raft_consensus.cc:2781] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:39.105764 29345 raft_consensus.cc:2454] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebba9f24fa6a45d89db719d5530fb661 in term 0.
I20260809 16:11:39.105809 29468 raft_consensus.cc:482] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:39.105839 29468 raft_consensus.cc:3037] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:39.106664 29468 raft_consensus.cc:504] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:39.106774 29468 leader_election.cc:288] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [CANDIDATE]: Term 1 election: Requested vote from peers 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130:40737), 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243)
I20260809 16:11:39.106941 29130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c284dbf63bcc40e7bab2458508cdb0fc" candidate_uuid: "ebba9f24fa6a45d89db719d5530fb661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "299f27b9a8e14c34bd2174cabd1efa3d"
I20260809 16:11:39.106953 29345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c284dbf63bcc40e7bab2458508cdb0fc" candidate_uuid: "ebba9f24fa6a45d89db719d5530fb661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592b82b92a0c4cc7af1edf8f657713d5"
I20260809 16:11:39.107014 29345 raft_consensus.cc:3037] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:39.107016 29130 raft_consensus.cc:3037] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:39.107687 29345 raft_consensus.cc:2454] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebba9f24fa6a45d89db719d5530fb661 in term 1.
I20260809 16:11:39.107746 29130 raft_consensus.cc:2454] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebba9f24fa6a45d89db719d5530fb661 in term 1.
I20260809 16:11:39.107867 28831 leader_election.cc:302] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 592b82b92a0c4cc7af1edf8f657713d5, ebba9f24fa6a45d89db719d5530fb661; no voters:
I20260809 16:11:39.107944 29468 raft_consensus.cc:2781] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:39.108078 29468 raft_consensus.cc:686] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 1 LEADER]: Becoming Leader. State: Replica: ebba9f24fa6a45d89db719d5530fb661, State: Running, Role: LEADER
I20260809 16:11:39.108165 29468 consensus_queue.cc:237] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } }
I20260809 16:11:39.109154 27440 catalog_manager.cc:5495] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129). New cstate: current_term: 1 leader_uuid: "ebba9f24fa6a45d89db719d5530fb661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:39.109661 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: NON_VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: true } } }
I20260809 16:11:39.109747 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
W20260809 16:11:39.110159 28999 tablet.cc:2369] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:39.110458 28350 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: attempting to promote NON_VOTER b9bc63a0403a4f2794d7c6248746385b to VOTER
I20260809 16:11:39.110567 28350 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } }
I20260809 16:11:39.110965 28430 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Refusing update from remote peer d4cb60585dc14321b9637dcc9412ddf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 16:11:39.111032 28739 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 LEARNER]: Refusing update from remote peer d4cb60585dc14321b9637dcc9412ddf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 16:11:39.111191 28664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 16:11:39.111300 28348 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 16:11:39.111928 28350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b9bc63a0403a4f2794d7c6248746385b (127.16.224.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.112073 28430 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b9bc63a0403a4f2794d7c6248746385b (127.16.224.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.112118 28739 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b9bc63a0403a4f2794d7c6248746385b (127.16.224.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.112818 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.112895 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:39.112947 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.112993 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:39.112953 28786 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.113022 28350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.113039 28786 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:11:39.113063 28350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:39.113214 29484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4cb60585dc14321b9637dcc9412ddf8" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:39.113299 29484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:11:39.117019 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.115034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.16.224.129:41065)
0809 16:11:39.115073 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:39.115075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:39.115081 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:11:39.115109 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:39.115110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:39.115123 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:39.115176 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:39.115264 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.115685 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.116430 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.116629 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.116673 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.116783 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:39.116881 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:39.116941 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:39.116960 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 16:11:39.116975 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:39.118610 29345 raft_consensus.cc:1261] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Refusing update from remote peer ebba9f24fa6a45d89db719d5530fb661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:39.118623 29130 raft_consensus.cc:1261] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 FOLLOWER]: Refusing update from remote peer ebba9f24fa6a45d89db719d5530fb661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:39.118901 29468 consensus_queue.cc:1035] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:39.119045 29478 consensus_queue.cc:1035] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:39.122103 29487 mvcc.cc:205] Tried to move back new op lower bound from 7316651618787483648 to 7316651618747707392. Current Snapshot: MvccSnapshot[applied={T|T < 7316651618787483648 or (T in {7316651618787483648})}]
I20260809 16:11:39.122787 29488 mvcc.cc:205] Tried to move back new op lower bound from 7316651618787483648 to 7316651618747707392. Current Snapshot: MvccSnapshot[applied={T|T < 7316651618787483648}]
I20260809 16:11:39.125486 27440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:39.125564 27440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:39.126955 27440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4cb60585dc14321b9637dcc9412ddf8: Sending DeleteTablet for 3 replicas of tablet c284dbf63bcc40e7bab2458508cdb0fc
I20260809 16:11:39.127275 29306 tablet_service.cc:1514] Processing DeleteTablet for tablet c284dbf63bcc40e7bab2458508cdb0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:39 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:57684
I20260809 16:11:39.127285 28895 tablet_service.cc:1514] Processing DeleteTablet for tablet c284dbf63bcc40e7bab2458508cdb0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:39 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:53508
I20260809 16:11:39.127285 29101 tablet_service.cc:1514] Processing DeleteTablet for tablet c284dbf63bcc40e7bab2458508cdb0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:39 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:43624
I20260809 16:11:39.127781 29502 tablet_replica.cc:330] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: stopping tablet replica
I20260809 16:11:39.127882 29502 raft_consensus.cc:2229] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:39.127981 29502 raft_consensus.cc:2258] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:39.128146 29501 tablet_replica.cc:330] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: stopping tablet replica
I20260809 16:11:39.128234 29501 raft_consensus.cc:2229] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:39.128350 29501 raft_consensus.cc:2258] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:39.128657 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 27393
I20260809 16:11:39.128691 29500 tablet_replica.cc:330] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: stopping tablet replica
I20260809 16:11:39.128775 29500 raft_consensus.cc:2229] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:39.128782 29501 ts_tablet_manager.cc:1905] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:39.128856 29500 raft_consensus.cc:2258] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:39.130182 29501 ts_tablet_manager.cc:1918] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:39.130236 29501 log.cc:1196] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/c284dbf63bcc40e7bab2458508cdb0fc
I20260809 16:11:39.130499 29501 ts_tablet_manager.cc:1939] T c284dbf63bcc40e7bab2458508cdb0fc P ebba9f24fa6a45d89db719d5530fb661: Deleting consensus metadata
I20260809 16:11:39.135607 29500 ts_tablet_manager.cc:1905] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:39.137079 29500 ts_tablet_manager.cc:1918] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:39.137140 29500 log.cc:1196] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/c284dbf63bcc40e7bab2458508cdb0fc
I20260809 16:11:39.137346 29500 ts_tablet_manager.cc:1939] T c284dbf63bcc40e7bab2458508cdb0fc P 592b82b92a0c4cc7af1edf8f657713d5: Deleting consensus metadata
I20260809 16:11:39.142758 29502 ts_tablet_manager.cc:1905] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:39.144523 29502 ts_tablet_manager.cc:1918] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:39.144611 29502 log.cc:1196] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/c284dbf63bcc40e7bab2458508cdb0fc
I20260809 16:11:39.144909 29502 ts_tablet_manager.cc:1939] T c284dbf63bcc40e7bab2458508cdb0fc P 299f27b9a8e14c34bd2174cabd1efa3d: Deleting consensus metadata
W20260809 16:11:39.147437 29237 connection.cc:535] server connection from 127.0.0.1:57684 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:11:39.147733 28829 connection.cc:535] server connection from 127.0.0.1:53508 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:11:39.147959 29038 connection.cc:535] server connection from 127.0.0.1:43624 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:11:39.151155 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.128152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34044 (local address 127.16.224.188:36765)
0809 16:11:39.128198 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:39.128200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:39.142684 (+ 14484us) server_negotiation.cc:407] Connection header received
0809 16:11:39.142826 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:39.142827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:39.142840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:39.142868 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:39.143705 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.144433 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.145485 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.145812 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.145887 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.147440 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:39.147442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:39.147442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:39.147451 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:39.149210 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:39.150172 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:39.150173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:39.150173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:39.150242 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:39.150491 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:39.150493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:39.150493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:39.150643 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:39.150719 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:39.150969 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 16:11:39.151060 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:39.156674 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.149774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34048 (local address 127.16.224.188:36765)
0809 16:11:39.149817 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:39.149818 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:39.150011 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:11:39.150104 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:39.150105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:39.150119 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:39.150161 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:39.151215 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.151872 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.152865 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.153183 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.153247 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.154300 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:39.154302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:39.154303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:39.154350 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:39.155700 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:39.156003 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:39.156004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:39.156004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:39.156059 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:39.156217 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:39.156218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:39.156218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:39.156322 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:39.156422 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:39.156500 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:11:39.156580 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:39.157236 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:39.149604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.16.224.189:39427)
0809 16:11:39.149661 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:39.149663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:39.149740 (+ 77us) server_negotiation.cc:407] Connection header received
0809 16:11:39.149851 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:39.149852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:39.149869 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:39.149898 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:39.150634 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.151464 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.152637 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:39.152939 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:39.152997 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:39.154470 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:39.154472 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:39.154472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:39.154480 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:39.155975 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:39.156348 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:39.156350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:39.156350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:39.156407 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:39.156591 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:39.156591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:39.156592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:39.156718 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:39.156794 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:39.157092 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 16:11:39.157172 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:11:40.123404 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.123073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.129:50925)
0809 16:11:40.123237 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.123342 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260809 16:11:40.123437 29520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.123073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.131:49901)
0809 16:11:40.123344 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.123403 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":76,"threads_started":1}
W20260809 16:11:40.123638 29420 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:42645 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
W20260809 16:11:40.123667 28996 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:42645 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
I20260809 16:11:40.124046 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.123715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:38021)
0809 16:11:40.123928 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.124006 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":63,"threads_started":1}
W20260809 16:11:40.124285 29205 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:42645 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
I20260809 16:11:40.627696 29523 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4cb60585dc14321b9637dcc9412ddf8)
I20260809 16:11:40.627707 29525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4cb60585dc14321b9637dcc9412ddf8)
I20260809 16:11:40.627821 29525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } }
I20260809 16:11:40.627835 29523 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } }
I20260809 16:11:40.628072 29523 leader_election.cc:288] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4cb60585dc14321b9637dcc9412ddf8 (127.16.224.190:42645), 6f296774bf534b358b473d832dea767f (127.16.224.189:39427)
I20260809 16:11:40.628075 29525 leader_election.cc:288] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4cb60585dc14321b9637dcc9412ddf8 (127.16.224.190:42645), b9bc63a0403a4f2794d7c6248746385b (127.16.224.188:36765)
I20260809 16:11:40.628537 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.628321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35884)
0809 16:11:40.628431 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.628495 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260809 16:11:40.628563 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.628164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35880)
0809 16:11:40.628490 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.628529 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":47,"threads_started":1}
W20260809 16:11:40.628650 28696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
W20260809 16:11:40.628651 28387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
I20260809 16:11:40.629077 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.628970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35898)
0809 16:11:40.629010 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.629045 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:11:40.629119 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.629037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35912)
0809 16:11:40.629073 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.629095 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:11:40.629186 28387 leader_election.cc:334] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4cb60585dc14321b9637dcc9412ddf8 (127.16.224.190:42645): Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
W20260809 16:11:40.629292 28696 leader_election.cc:334] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4cb60585dc14321b9637dcc9412ddf8 (127.16.224.190:42645): Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
I20260809 16:11:40.631592 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.628219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:36765 (local address 127.0.0.1:34062)
0809 16:11:40.628325 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.628335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:40.628388 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:40.628729 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:40.628730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:40.628732 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:40.628789 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.628791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.629679 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.629680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.630821 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.630824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.631410 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.631411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.631462 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.631466 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:40.631510 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:11:40.631546 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":49,"threads_started":1}
I20260809 16:11:40.631623 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.628421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.16.224.188:36765)
0809 16:11:40.628520 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.628522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.628529 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:40.628553 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.628554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.628564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.628634 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:40.628861 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.629593 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.630894 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.631328 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.631399 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.631408 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.631524 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.631561 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":59,"threads_started":1}
I20260809 16:11:40.631906 28739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f296774bf534b358b473d832dea767f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b9bc63a0403a4f2794d7c6248746385b" is_pre_election: true
I20260809 16:11:40.632020 28739 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f296774bf534b358b473d832dea767f in term 1.
I20260809 16:11:40.632026 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.628236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39427 (local address 127.0.0.1:38654)
0809 16:11:40.628848 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.628858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:40.629007 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:40.629291 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:40.629292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:40.629295 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:40.629333 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.629335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.630027 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.630028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.631105 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.631107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.631840 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.631841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.631896 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.631900 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:40.631939 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 16:11:40.631968 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":43,"threads_started":1}
I20260809 16:11:40.632062 29535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.628361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.16.224.189:39427)
0809 16:11:40.628701 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.628703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.629150 (+ 447us) server_negotiation.cc:407] Connection header received
0809 16:11:40.629174 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.629175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.629184 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.629250 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:40.629393 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.629949 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.631171 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.631702 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.631762 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.631777 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.631967 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.632006 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":34,"threads_started":1}
I20260809 16:11:40.632203 28388 leader_election.cc:302] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f296774bf534b358b473d832dea767f, b9bc63a0403a4f2794d7c6248746385b; no voters: d4cb60585dc14321b9637dcc9412ddf8
I20260809 16:11:40.632320 29525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 16:11:40.632274 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9bc63a0403a4f2794d7c6248746385b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f296774bf534b358b473d832dea767f" is_pre_election: true
I20260809 16:11:40.632360 29525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d4cb60585dc14321b9637dcc9412ddf8)
I20260809 16:11:40.632380 29525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:11:40.633072 29525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } }
I20260809 16:11:40.633222 29525 leader_election.cc:288] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [CANDIDATE]: Term 2 election: Requested vote from peers d4cb60585dc14321b9637dcc9412ddf8 (127.16.224.190:42645), b9bc63a0403a4f2794d7c6248746385b (127.16.224.188:36765)
I20260809 16:11:40.633245 28430 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b9bc63a0403a4f2794d7c6248746385b in current term 2: Already voted for candidate 6f296774bf534b358b473d832dea767f in this term.
I20260809 16:11:40.633397 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.633293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35916)
0809 16:11:40.633341 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.633376 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 16:11:40.633392 28739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f296774bf534b358b473d832dea767f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b9bc63a0403a4f2794d7c6248746385b"
I20260809 16:11:40.633471 28695 leader_election.cc:302] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9bc63a0403a4f2794d7c6248746385b; no voters: 6f296774bf534b358b473d832dea767f, d4cb60585dc14321b9637dcc9412ddf8
I20260809 16:11:40.633471 28739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:11:40.633665 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.633588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35926)
0809 16:11:40.633622 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.633642 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:11:40.633728 28387 leader_election.cc:334] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d4cb60585dc14321b9637dcc9412ddf8 (127.16.224.190:42645): Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
I20260809 16:11:40.634147 28739 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f296774bf534b358b473d832dea767f in term 2.
I20260809 16:11:40.634223 29523 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:11:40.634341 28388 leader_election.cc:302] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f296774bf534b358b473d832dea767f, b9bc63a0403a4f2794d7c6248746385b; no voters: d4cb60585dc14321b9637dcc9412ddf8
I20260809 16:11:40.634433 29525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:11:40.634577 29525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [term 2 LEADER]: Becoming Leader. State: Replica: 6f296774bf534b358b473d832dea767f, State: Running, Role: LEADER
I20260809 16:11:40.634645 29525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } }
I20260809 16:11:40.634949 29540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f296774bf534b358b473d832dea767f. Latest consensus state: current_term: 2 leader_uuid: "6f296774bf534b358b473d832dea767f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:40.635042 29540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:40.635215 29544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:40.635741 29544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:40.635812 29544 catalog_manager.cc:1259] Loaded cluster ID: 4743acfe20e34f9795156e1abdc3e193
I20260809 16:11:40.635833 29544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:40.635960 29544 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:40.636013 29544 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Loaded TSK: 0
I20260809 16:11:40.636106 29544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:11:40.684108 28409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:40.685766 28409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:40.686838 28739 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [term 2 FOLLOWER]: Refusing update from remote peer 6f296774bf534b358b473d832dea767f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 16:11:40.686873 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.686744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35940)
0809 16:11:40.686796 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.686841 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:11:40.687091 29525 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 16:11:40.687244 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.687143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.0.0.1:35950)
0809 16:11:40.687191 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.687222 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:11:40.687315 28387 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f -> Peer d4cb60585dc14321b9637dcc9412ddf8 (127.16.224.190:42645): Couldn't send request to peer d4cb60585dc14321b9637dcc9412ddf8. Status: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 16:11:40.687752 29523 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f296774bf534b358b473d832dea767f. Latest consensus state: current_term: 2 leader_uuid: "6f296774bf534b358b473d832dea767f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:40.687846 29523 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:11:40.688252 29525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6f296774bf534b358b473d832dea767f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:40.688337 29525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:40.688421 29523 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6f296774bf534b358b473d832dea767f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d4cb60585dc14321b9637dcc9412ddf8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42645 } } peers { permanent_uuid: "6f296774bf534b358b473d832dea767f" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39427 } attrs { promote: false } } peers { permanent_uuid: "b9bc63a0403a4f2794d7c6248746385b" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 36765 } attrs { promote: false } } }
I20260809 16:11:40.688485 29547 mvcc.cc:205] Tried to move back new op lower bound from 7316651625211305984 to 7316651625000022016. Current Snapshot: MvccSnapshot[applied={T|T < 7316651625211305984}]
I20260809 16:11:40.688500 29523 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc63a0403a4f2794d7c6248746385b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:11:40.693328 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.690148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:41065 (local address 127.0.0.1:53526)
0809 16:11:40.690288 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.690297 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:40.690414 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:40.690560 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:40.690561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:40.690562 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:40.690613 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.690615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.691404 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.691405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.692390 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.692392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.693122 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.693123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.693178 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.693183 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:11:40.693226 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:11:40.693265 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 16:11:40.693408 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.690090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:42243 (local address 127.0.0.1:57700)
0809 16:11:40.690183 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.690193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:40.690240 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:40.690653 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:40.690654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:40.690657 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:40.690718 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.690719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.691651 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.691652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.692541 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.692544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.693202 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.693203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.693266 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.693269 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:40.693324 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:11:40.693358 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 16:11:40.693435 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.690207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57700 (local address 127.16.224.131:42243)
0809 16:11:40.690344 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.690346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.690356 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:40.690384 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.690385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.690396 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.690599 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:40.690798 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.691560 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.692629 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.693139 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.693206 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.693213 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.693329 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.693365 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":51,"threads_started":1}
I20260809 16:11:40.693535 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.690204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.16.224.129:41065)
0809 16:11:40.690356 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.690358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.690367 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:40.690490 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.690491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.690500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.690667 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:40.690706 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.691326 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.692504 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.693069 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.693128 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.693137 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.693415 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.693460 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260809 16:11:40.693732 29306 tablet_service.cc:1467] Processing CreateTablet for tablet 622c16e907d946ae976b5689d8f48250 (DEFAULT_TABLE table=test-table [id=9d1de1ce3d3c49bebdcf42350de1ec17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:40.693882 29306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622c16e907d946ae976b5689d8f48250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:40.693863 28895 tablet_service.cc:1467] Processing CreateTablet for tablet 622c16e907d946ae976b5689d8f48250 (DEFAULT_TABLE table=test-table [id=9d1de1ce3d3c49bebdcf42350de1ec17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:40.693970 28895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622c16e907d946ae976b5689d8f48250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:40.694347 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.690163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:40737 (local address 127.0.0.1:43640)
0809 16:11:40.690346 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.690356 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:40.690407 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:40.690971 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:40.690972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:40.690974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:40.691028 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.691030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.691905 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.691906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.692969 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.692972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.694087 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.694088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.694161 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.694165 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:40.694221 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:11:40.694258 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260809 16:11:40.695034 29561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.690429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.16.224.130:40737)
0809 16:11:40.690839 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.690841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.690848 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:40.690873 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.690873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.690884 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.691088 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:40.691125 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.691827 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.693049 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.693995 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.694072 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.694081 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.694231 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.694270 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":142,"threads_started":1}
I20260809 16:11:40.695325 29563 tablet_bootstrap.cc:492] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: Bootstrap starting.
I20260809 16:11:40.695390 29101 tablet_service.cc:1467] Processing CreateTablet for tablet 622c16e907d946ae976b5689d8f48250 (DEFAULT_TABLE table=test-table [id=9d1de1ce3d3c49bebdcf42350de1ec17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:40.695492 29101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622c16e907d946ae976b5689d8f48250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:40.695617 29563 tablet_bootstrap.cc:654] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:40.696077 29563 tablet_bootstrap.cc:492] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: No bootstrap required, opened a new log
I20260809 16:11:40.696111 29563 ts_tablet_manager.cc:1397] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:40.696215 29563 raft_consensus.cc:348] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.696280 29563 raft_consensus.cc:374] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:40.696306 29563 raft_consensus.cc:729] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 592b82b92a0c4cc7af1edf8f657713d5, State: Initialized, Role: FOLLOWER
I20260809 16:11:40.696348 29563 consensus_queue.cc:260] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.696507 29568 tablet_bootstrap.cc:492] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: Bootstrap starting.
I20260809 16:11:40.696475 29563 ts_tablet_manager.cc:1428] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:40.696890 29568 tablet_bootstrap.cc:654] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:40.697093 29564 tablet_bootstrap.cc:492] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661: Bootstrap starting.
I20260809 16:11:40.697314 29424 heartbeater.cc:503] Master 127.16.224.189:39427 was elected leader, sending a full tablet report...
I20260809 16:11:40.697486 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.696773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.131:39037)
0809 16:11:40.697269 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.697314 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":76,"threads_started":1}
I20260809 16:11:40.697515 29564 tablet_bootstrap.cc:654] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:40.697644 29568 tablet_bootstrap.cc:492] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: No bootstrap required, opened a new log
I20260809 16:11:40.697696 29568 ts_tablet_manager.cc:1397] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:40.697824 29568 raft_consensus.cc:348] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.697885 29568 raft_consensus.cc:374] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:40.697911 29568 raft_consensus.cc:729] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299f27b9a8e14c34bd2174cabd1efa3d, State: Initialized, Role: FOLLOWER
I20260809 16:11:40.697906 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.697669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.131:47731)
0809 16:11:40.697858 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.697883 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":82,"threads_started":1}
I20260809 16:11:40.697968 29568 consensus_queue.cc:260] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.697991 29564 tablet_bootstrap.cc:492] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661: No bootstrap required, opened a new log
I20260809 16:11:40.698053 29564 ts_tablet_manager.cc:1397] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:40.698154 29568 ts_tablet_manager.cc:1428] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:40.698153 29564 raft_consensus.cc:348] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.698210 29564 raft_consensus.cc:374] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:40.698228 29564 raft_consensus.cc:729] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebba9f24fa6a45d89db719d5530fb661, State: Initialized, Role: FOLLOWER
I20260809 16:11:40.698261 29564 consensus_queue.cc:260] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.698423 29564 ts_tablet_manager.cc:1428] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:40.698637 29206 heartbeater.cc:503] Master 127.16.224.189:39427 was elected leader, sending a full tablet report...
I20260809 16:11:40.698995 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.698723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.129:38127)
0809 16:11:40.698926 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.698959 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260809 16:11:40.699184 28997 heartbeater.cc:503] Master 127.16.224.189:39427 was elected leader, sending a full tablet report...
I20260809 16:11:40.699255 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.698274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:34409)
0809 16:11:40.699178 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.699224 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":98,"threads_started":1}
I20260809 16:11:40.699414 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.699259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.129:60955)
0809 16:11:40.699337 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.699388 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 16:11:40.699679 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.699516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:55251)
0809 16:11:40.699628 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.699656 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":69,"threads_started":1}
I20260809 16:11:40.787158 29575 raft_consensus.cc:482] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:40.787261 29575 raft_consensus.cc:504] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.787612 29575 leader_election.cc:288] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243), ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129:41065)
I20260809 16:11:40.790230 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.787854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:45407 (local address 127.16.224.129:41065)
0809 16:11:40.787916 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.787918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.787927 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:40.787954 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.787955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.787965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.788013 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:40.788135 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.788634 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.789572 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.789927 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.789987 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.789998 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.790128 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.790168 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:11:40.790230 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.787777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:41065 (local address 127.16.224.130:45407)
0809 16:11:40.787828 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.787840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:40.787893 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:40.788023 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:40.788024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:40.788027 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:40.788082 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.788085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.788710 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.788711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.789493 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.789495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.789998 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.790000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.790068 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.790072 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:40.790115 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:11:40.790151 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":18}
I20260809 16:11:40.790361 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.787785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:42243 (local address 127.16.224.130:49211)
0809 16:11:40.787965 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.787975 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:40.788039 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:40.788191 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:40.788192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:40.788194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:40.788247 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.788248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.788856 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.788857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.789583 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:40.789585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.790101 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.790102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:40.790167 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.790171 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:40.790222 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:11:40.790260 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":126}
I20260809 16:11:40.790587 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.787883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:49211 (local address 127.16.224.131:42243)
0809 16:11:40.787934 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.787936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.788035 (+ 99us) server_negotiation.cc:407] Connection header received
0809 16:11:40.788110 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.788111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.788122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.788170 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:40.788308 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.788775 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.789670 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.790026 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.790094 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.790104 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.790233 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.790281 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:40.790695 28930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622c16e907d946ae976b5689d8f48250" candidate_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebba9f24fa6a45d89db719d5530fb661" is_pre_election: true
I20260809 16:11:40.790786 29345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622c16e907d946ae976b5689d8f48250" candidate_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592b82b92a0c4cc7af1edf8f657713d5" is_pre_election: true
I20260809 16:11:40.790859 28930 raft_consensus.cc:2454] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 299f27b9a8e14c34bd2174cabd1efa3d in term 0.
I20260809 16:11:40.790889 29345 raft_consensus.cc:2454] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 299f27b9a8e14c34bd2174cabd1efa3d in term 0.
I20260809 16:11:40.791049 29038 leader_election.cc:302] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 299f27b9a8e14c34bd2174cabd1efa3d, 592b82b92a0c4cc7af1edf8f657713d5; no voters:
I20260809 16:11:40.791231 29575 raft_consensus.cc:2781] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:40.791272 29575 raft_consensus.cc:482] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:40.791290 29575 raft_consensus.cc:3037] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:40.791934 29575 raft_consensus.cc:504] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.792053 29575 leader_election.cc:288] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [CANDIDATE]: Term 1 election: Requested vote from peers 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243), ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129:41065)
I20260809 16:11:40.792220 29345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622c16e907d946ae976b5689d8f48250" candidate_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592b82b92a0c4cc7af1edf8f657713d5"
I20260809 16:11:40.792250 28930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622c16e907d946ae976b5689d8f48250" candidate_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebba9f24fa6a45d89db719d5530fb661"
I20260809 16:11:40.792315 28930 raft_consensus.cc:3037] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:40.792313 29345 raft_consensus.cc:3037] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:40.792893 28930 raft_consensus.cc:2454] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 299f27b9a8e14c34bd2174cabd1efa3d in term 1.
I20260809 16:11:40.792970 29345 raft_consensus.cc:2454] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 299f27b9a8e14c34bd2174cabd1efa3d in term 1.
I20260809 16:11:40.793042 29036 leader_election.cc:302] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 299f27b9a8e14c34bd2174cabd1efa3d, ebba9f24fa6a45d89db719d5530fb661; no voters:
I20260809 16:11:40.793112 29575 raft_consensus.cc:2781] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:40.793282 29575 raft_consensus.cc:686] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 LEADER]: Becoming Leader. State: Replica: 299f27b9a8e14c34bd2174cabd1efa3d, State: Running, Role: LEADER
I20260809 16:11:40.793413 29575 consensus_queue.cc:237] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } }
I20260809 16:11:40.793613 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.793516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:50699)
0809 16:11:40.793555 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.793586 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 16:11:40.794118 28409 catalog_manager.cc:5495] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130). New cstate: current_term: 1 leader_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "299f27b9a8e14c34bd2174cabd1efa3d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 40737 } health_report { overall_health: HEALTHY } } }
I20260809 16:11:40.808859 29561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.806271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.16.224.130:40737)
0809 16:11:40.806333 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:40.806335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:40.806385 (+ 50us) server_negotiation.cc:407] Connection header received
0809 16:11:40.806472 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:40.806472 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:40.806493 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:40.806559 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:40.806705 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.807311 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.808069 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:40.808252 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:40.808292 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:40.808432 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:40.808545 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:40.808589 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:40.808775 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 16:11:40.808795 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:11:40.812628 29345 raft_consensus.cc:1261] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Refusing update from remote peer 299f27b9a8e14c34bd2174cabd1efa3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:40.812736 28930 raft_consensus.cc:1261] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 1 FOLLOWER]: Refusing update from remote peer 299f27b9a8e14c34bd2174cabd1efa3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:40.812894 29575 consensus_queue.cc:1035] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "592b82b92a0c4cc7af1edf8f657713d5" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 42243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:40.813047 29584 consensus_queue.cc:1035] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebba9f24fa6a45d89db719d5530fb661" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:40.813820 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.813688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.131:55109)
0809 16:11:40.813742 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.813778 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 16:11:40.814133 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.814038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.129:59539)
0809 16:11:40.814083 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.814107 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:11:40.815229 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.815130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:44945)
0809 16:11:40.815173 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.815199 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:11:40.815594 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.815507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.131:39409)
0809 16:11:40.815543 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.815566 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 16:11:40.816174 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.816094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.130:60717)
0809 16:11:40.816124 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.816149 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 16:11:40.817726 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.817638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42645 (local address 127.16.224.129:50525)
0809 16:11:40.817669 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.817700 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 16:11:40.818378 28409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:40.818483 28409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:40.819764 28409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f296774bf534b358b473d832dea767f: Sending DeleteTablet for 3 replicas of tablet 622c16e907d946ae976b5689d8f48250
I20260809 16:11:40.820088 28895 tablet_service.cc:1514] Processing DeleteTablet for tablet 622c16e907d946ae976b5689d8f48250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:40 UTC) from {username='oryx', principal='oryx/127.16.224.189@KRBTEST.COM'} at 127.0.0.1:53526
I20260809 16:11:40.820119 29306 tablet_service.cc:1514] Processing DeleteTablet for tablet 622c16e907d946ae976b5689d8f48250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:40 UTC) from {username='oryx', principal='oryx/127.16.224.189@KRBTEST.COM'} at 127.0.0.1:57700
I20260809 16:11:40.820283 29601 tablet_replica.cc:330] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661: stopping tablet replica
I20260809 16:11:40.820340 29601 raft_consensus.cc:2229] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:40.820402 29601 raft_consensus.cc:2258] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:40.820391 29101 tablet_service.cc:1514] Processing DeleteTablet for tablet 622c16e907d946ae976b5689d8f48250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:40 UTC) from {username='oryx', principal='oryx/127.16.224.189@KRBTEST.COM'} at 127.0.0.1:43640
I20260809 16:11:40.820787 29601 ts_tablet_manager.cc:1905] T 622c16e907d946ae976b5689d8f48250 P ebba9f24fa6a45d89db719d5530fb661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:40.820922 29603 tablet_replica.cc:330] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: stopping tablet replica
I20260809 16:11:40.821012 29603 raft_consensus.cc:2229] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:40.821147 29603 raft_consensus.cc:2258] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:40.821425 29603 ts_tablet_manager.cc:1905] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:40.821887 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 28660
I20260809 16:11:40.828048 29603 ts_tablet_manager.cc:1918] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:40.828168 29603 log.cc:1196] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/622c16e907d946ae976b5689d8f48250
I20260809 16:11:40.828441 29603 ts_tablet_manager.cc:1939] T 622c16e907d946ae976b5689d8f48250 P 299f27b9a8e14c34bd2174cabd1efa3d: Deleting consensus metadata
I20260809 16:11:40.831215 29602 tablet_replica.cc:330] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: stopping tablet replica
I20260809 16:11:40.831288 29602 raft_consensus.cc:2229] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:40.831344 29602 raft_consensus.cc:2258] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:40.831633 29602 ts_tablet_manager.cc:1905] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:40.833505 29602 ts_tablet_manager.cc:1918] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:40.833580 29602 log.cc:1196] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/622c16e907d946ae976b5689d8f48250
I20260809 16:11:40.833788 29602 ts_tablet_manager.cc:1939] T 622c16e907d946ae976b5689d8f48250 P 592b82b92a0c4cc7af1edf8f657713d5: Deleting consensus metadata
I20260809 16:11:40.833818 28389 catalog_manager.cc:4841] TS 299f27b9a8e14c34bd2174cabd1efa3d (127.16.224.130:40737): tablet 622c16e907d946ae976b5689d8f48250 (table test-table [id=9d1de1ce3d3c49bebdcf42350de1ec17]) successfully deleted
I20260809 16:11:40.834578 28389 catalog_manager.cc:4841] TS 592b82b92a0c4cc7af1edf8f657713d5 (127.16.224.131:42243): tablet 622c16e907d946ae976b5689d8f48250 (table test-table [id=9d1de1ce3d3c49bebdcf42350de1ec17]) successfully deleted
I20260809 16:11:40.840974 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:40.840845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:41065 (local address 127.0.0.1:53528)
0809 16:11:40.840902 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:40.840934 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:11:40.841131 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 29005
W20260809 16:11:40.841141 28387 catalog_manager.cc:4584] TS ebba9f24fa6a45d89db719d5530fb661 (127.16.224.129:41065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 622c16e907d946ae976b5689d8f48250: Network error: Client connection negotiation failed: client connection to 127.16.224.129:41065: connect: Connection refused (error 111)
I20260809 16:11:40.854308 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 29211
I20260809 16:11:40.862775 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 28191
I20260809 16:11:40.868880 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 28510
2026-08-09T16:11:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4787 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:40 dist-test-slave-fgb3 krb5kdc[29612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:40 dist-test-slave-fgb3 krb5kdc[29612](info): set up 2 sockets
Aug 09 16:11:40 dist-test-slave-fgb3 krb5kdc[29612](info): commencing operation
krb5kdc: starting...
W20260809 16:11:42.003834 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab.
Entry for principal oryx/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab.
I20260809 16:11:42.126807 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:34213
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab
--principal=oryx/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:34213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:42.207861 29640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:42.208065 29640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:42.208087 29640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:42.209209 29640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:42.209249 29640 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:11:42.209262 29640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:42.209285 29640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:42.209306 29640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:42.209327 29640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:42.210850 29640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:34213
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.190.keytab
--principal=oryx/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:34213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:42.211076 29640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:42.211304 29640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:42.215806 29649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.215801 29653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.215900 29650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:42.215960 29640 server_base.cc:1034] running on GCE node
I20260809 16:11:42.216221 29640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:42.216502 29640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:42.217639 29640 hybrid_clock.cc:648] HybridClock initialized: now 1786291902217623 us; error 28 us; skew 500 ppm
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:42.219959 29640 init.cc:377] Logged in from keytab as oryx/127.16.224.190@KRBTEST.COM (short username oryx)
I20260809 16:11:42.220377 29640 webserver.cc:460] Webserver started at http://127.16.224.190:37733/ using document root <none> and password file <none>
I20260809 16:11:42.220582 29640 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:42.220628 29640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:42.220752 29640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:42.221735 29640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "522313375d2e40f6a07264f5e37f31bc"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "b8f49e32338e621f7bb2fb84943acd1a"
server_key_iv: "fe77866ada2b2e20fc1db03473101e9b"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.222077 29640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "522313375d2e40f6a07264f5e37f31bc"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "b8f49e32338e621f7bb2fb84943acd1a"
server_key_iv: "fe77866ada2b2e20fc1db03473101e9b"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.223348 29640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:42.223996 29663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.224143 29640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:42.224229 29640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "522313375d2e40f6a07264f5e37f31bc"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "b8f49e32338e621f7bb2fb84943acd1a"
server_key_iv: "fe77866ada2b2e20fc1db03473101e9b"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.224294 29640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:42.233644 29640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:42.235260 29640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:42.235385 29640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:42.239796 29640 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:34213
I20260809 16:11:42.239851 29768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:34213 every 8 connection(s)
I20260809 16:11:42.240139 29640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:42.240749 29771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.241498 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 29640
I20260809 16:11:42.241600 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:42.241801 17282 external_mini_cluster.cc:1428] Setting key 92deb41819a448355198d1aebe10e730
I20260809 16:11:42.243613 29771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Bootstrap starting.
I20260809 16:11:42.244210 29771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.244488 29771 log.cc:824] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:42.245096 29771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: No bootstrap required, opened a new log
I20260809 16:11:42.246981 29771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522313375d2e40f6a07264f5e37f31bc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 34213 } }
I20260809 16:11:42.247115 29771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.247148 29771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522313375d2e40f6a07264f5e37f31bc, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.247274 29771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522313375d2e40f6a07264f5e37f31bc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 34213 } }
I20260809 16:11:42.247346 29771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:42.247378 29771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:42.247426 29771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 0 FOLLOWER]: Advancing to term 1
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:42.247951 29771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522313375d2e40f6a07264f5e37f31bc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 34213 } }
I20260809 16:11:42.248051 29771 leader_election.cc:302] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 522313375d2e40f6a07264f5e37f31bc; no voters:
I20260809 16:11:42.248203 29771 leader_election.cc:288] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:42.248235 29776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:42.248346 29776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [term 1 LEADER]: Becoming Leader. State: Replica: 522313375d2e40f6a07264f5e37f31bc, State: Running, Role: LEADER
I20260809 16:11:42.248469 29771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:42.248456 29776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522313375d2e40f6a07264f5e37f31bc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 34213 } }
I20260809 16:11:42.248816 29778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 522313375d2e40f6a07264f5e37f31bc. Latest consensus state: current_term: 1 leader_uuid: "522313375d2e40f6a07264f5e37f31bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522313375d2e40f6a07264f5e37f31bc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 34213 } } }
I20260809 16:11:42.248901 29778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:42.249081 29777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "522313375d2e40f6a07264f5e37f31bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "522313375d2e40f6a07264f5e37f31bc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 34213 } } }
I20260809 16:11:42.249122 29789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:42.249141 29777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:42.249706 29789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:42.251574 29789 catalog_manager.cc:1347] Generated new cluster ID: 028b639e0c454efd82c2021868e57ccc
I20260809 16:11:42.251632 29789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:42.252347 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.242299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.16.224.190:34213)
0809 16:11:42.242513 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.242521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.242545 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:11:42.242951 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.242957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.243003 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.243122 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.243767 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.244721 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.245516 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.245919 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.246014 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.248241 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.248245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.248245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.248258 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.250235 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.251004 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.251008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.251008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.251235 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.251436 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.251437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.251437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.251738 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.251878 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.252004 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.252141 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260809 16:11:42.264137 29789 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:42.264652 29789 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:42.268872 29789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Generated new TSK 0
I20260809 16:11:42.269037 29789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab.
Entry for principal oryx/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab.
I20260809 16:11:42.294826 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:34213
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab
--principal=oryx/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:42.376986 29817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:42.377183 29817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:42.377205 29817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:42.378374 29817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:42.378415 29817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:42.378436 29817 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:42.378465 29817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:42.379988 29817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab
--principal=oryx/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:34213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:42.380215 29817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:42.380431 29817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:42.385098 29825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.385124 29823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.385126 29822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:42.385387 29817 server_base.cc:1034] running on GCE node
I20260809 16:11:42.385596 29817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:42.385807 29817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:42.386943 29817 hybrid_clock.cc:648] HybridClock initialized: now 1786291902386927 us; error 31 us; skew 500 ppm
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:42.389413 29817 init.cc:377] Logged in from keytab as oryx/127.16.224.129@KRBTEST.COM (short username oryx)
I20260809 16:11:42.389842 29817 webserver.cc:460] Webserver started at http://127.16.224.129:44599/ using document root <none> and password file <none>
I20260809 16:11:42.390045 29817 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:42.390084 29817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:42.390177 29817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:42.391211 29817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "7fa0096f42e04394942012c0177703d7"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "c7449876987fa147bd01e354a5d5cf86"
server_key_iv: "07864aa1f8a5ca8f19d25a242644db32"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.391522 29817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "7fa0096f42e04394942012c0177703d7"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "c7449876987fa147bd01e354a5d5cf86"
server_key_iv: "07864aa1f8a5ca8f19d25a242644db32"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.392782 29817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:11:42.393416 29835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.393559 29817 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:42.393640 29817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "7fa0096f42e04394942012c0177703d7"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "c7449876987fa147bd01e354a5d5cf86"
server_key_iv: "07864aa1f8a5ca8f19d25a242644db32"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.393704 29817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:42.414983 29817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:42.417047 29817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:42.417176 29817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:42.417704 29817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:42.417747 29817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.417775 29817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:42.417793 29817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:42.424710 29817 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:44729
I20260809 16:11:42.424796 30049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:44729 every 8 connection(s)
I20260809 16:11:42.425071 29817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:42.426946 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.418097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:47233 (local address 127.16.224.190:34213)
0809 16:11:42.418165 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.418168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.418345 (+ 177us) server_negotiation.cc:407] Connection header received
0809 16:11:42.418845 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.418847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.418861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.418936 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.419551 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.420313 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.421356 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.421671 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.421748 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.424165 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.424166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.424167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.424175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.425681 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.426067 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.426068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.426069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.426130 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.426376 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.426377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.426378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.426504 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.426567 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.426808 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.426881 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:11:42.427088 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.418051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:34213 (local address 127.16.224.129:47233)
0809 16:11:42.418199 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.418240 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.418745 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.419016 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.419020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.419279 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:42.419469 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.419474 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.420401 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.420402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.421283 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.421287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.421767 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.421768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.421885 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.422333 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:42.422344 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:42.424083 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:42.425754 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.425762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.425764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.425998 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.426206 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.426206 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.426207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.426303 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.426560 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:42.426564 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:42.426695 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.426772 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.426900 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260809 16:11:42.427558 30050 heartbeater.cc:347] Connected to a master server at 127.16.224.190:34213
I20260809 16:11:42.427639 30050 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:42.427839 30050 heartbeater.cc:511] Master 127.16.224.190:34213 requested a full tablet report, sending...
I20260809 16:11:42.428363 29687 ts_manager.cc:194] Registered new tserver with Master: 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729)
I20260809 16:11:42.429034 29687 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:47233
I20260809 16:11:42.430563 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 29817
I20260809 16:11:42.430662 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:42.430874 17282 external_mini_cluster.cc:1428] Setting key ed6eb25cb2558b6d972bc97e8fffe5ac
I20260809 16:11:42.433545 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.428606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:48273 (local address 127.16.224.190:34213)
0809 16:11:42.428646 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.428648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.428730 (+ 82us) server_negotiation.cc:407] Connection header received
0809 16:11:42.428787 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.428788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.428797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.428841 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.429163 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.429654 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.430348 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.430556 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.430610 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.431464 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.431465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.431466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.431473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.432581 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.432875 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.432876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.432876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.432929 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.433086 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.433087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.433087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.433189 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.433243 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.433385 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.433460 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:11:42.433560 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.428533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:34213 (local address 127.16.224.129:48273)
0809 16:11:42.428668 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.428678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.428735 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.428843 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.428844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.428999 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:42.429050 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.429051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.429743 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.429744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.430263 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.430265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.430616 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.430617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.430684 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.431069 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:42.431074 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:42.431388 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:42.432654 (+ 1266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.432655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.432656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.432809 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.432981 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.432982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.432982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.433037 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.433242 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:42.433242 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:42.433288 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.433356 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.433462 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab.
Entry for principal oryx/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab.
I20260809 16:11:42.453794 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:34213
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab
--principal=oryx/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:42.534626 30067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:42.534828 30067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:42.534860 30067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:42.535965 30067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:42.536005 30067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:42.536031 30067 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:42.536051 30067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:42.537530 30067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab
--principal=oryx/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:34213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:42.537767 30067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:42.537981 30067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:42.542745 30078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.542769 30084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.542745 30080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:42.543053 30067 server_base.cc:1034] running on GCE node
I20260809 16:11:42.543268 30067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:42.543505 30067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:42.544647 30067 hybrid_clock.cc:648] HybridClock initialized: now 1786291902544635 us; error 31 us; skew 500 ppm
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:42.547206 30067 init.cc:377] Logged in from keytab as oryx/127.16.224.130@KRBTEST.COM (short username oryx)
I20260809 16:11:42.547683 30067 webserver.cc:460] Webserver started at http://127.16.224.130:35953/ using document root <none> and password file <none>
I20260809 16:11:42.547896 30067 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:42.547946 30067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:42.548085 30067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:42.549216 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "0946bd92b7984c2b97cfa60d287fec02"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "d29b13abcd4b1cd5e02355a778a55500"
server_key_iv: "58a1e71d31f6fb4964b6e9721678165e"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.549585 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "0946bd92b7984c2b97cfa60d287fec02"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "d29b13abcd4b1cd5e02355a778a55500"
server_key_iv: "58a1e71d31f6fb4964b6e9721678165e"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.550961 30067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:11:42.551698 30093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.551873 30067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:42.551967 30067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "0946bd92b7984c2b97cfa60d287fec02"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "d29b13abcd4b1cd5e02355a778a55500"
server_key_iv: "58a1e71d31f6fb4964b6e9721678165e"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.552030 30067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:42.560607 30067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:42.562798 30067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:42.562948 30067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:42.563472 30067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:42.563514 30067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.563534 30067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:42.563547 30067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.569769 30067 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:44411
I20260809 16:11:42.569875 30295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:44411 every 8 connection(s)
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:42.570138 30067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:42.572849 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.563865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:43499 (local address 127.16.224.190:34213)
0809 16:11:42.563935 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.563938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.564102 (+ 164us) server_negotiation.cc:407] Connection header received
0809 16:11:42.564593 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.564595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.564609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.564687 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.565498 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.566236 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.567802 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.568089 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.568195 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.570638 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.570640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.570640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.570647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.571862 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.572186 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.572186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.572187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.572231 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.572398 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.572399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.572399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.572490 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.572535 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.572733 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.572787 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:11:42.573014 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.563831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:34213 (local address 127.16.224.130:43499)
0809 16:11:42.563974 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.564016 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.564503 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.564771 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.564774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.565106 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:42.565393 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.565400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.566418 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.566420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.567708 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.567712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.568200 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.568201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.568326 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.568785 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:42.568796 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:42.570566 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:42.571927 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.571930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.571932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.572130 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.572273 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.572273 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.572274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.572358 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.572532 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:42.572534 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:42.572641 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.572711 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.572821 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260809 16:11:42.573472 30296 heartbeater.cc:347] Connected to a master server at 127.16.224.190:34213
I20260809 16:11:42.573560 30296 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:42.573747 30296 heartbeater.cc:511] Master 127.16.224.190:34213 requested a full tablet report, sending...
I20260809 16:11:42.574168 29688 ts_manager.cc:194] Registered new tserver with Master: 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411)
I20260809 16:11:42.574663 29688 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:43499
I20260809 16:11:42.578451 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 30067
I20260809 16:11:42.578549 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:11:42.578729 17282 external_mini_cluster.cc:1428] Setting key f8b13981e76136ffca097f8d528f7f2a
I20260809 16:11:42.579609 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.574575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:34213 (local address 127.16.224.130:53911)
0809 16:11:42.574694 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.574704 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.574762 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.574897 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.574898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.575084 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:42.575153 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.575156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.575728 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.575729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.576243 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.576244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.576521 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.576522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.576566 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.576869 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:42.576874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:42.577212 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:42.578598 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.578599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.578599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.578753 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.579024 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.579025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.579025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.579082 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.579369 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:42.579369 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:42.579407 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.579461 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.579542 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
I20260809 16:11:42.579653 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.574635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:53911 (local address 127.16.224.190:34213)
0809 16:11:42.574675 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.574677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.574747 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:11:42.574815 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.574817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.574827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.574890 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.575223 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.575652 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.576297 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.576474 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.576517 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.577286 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.577287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.577287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.577293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.578511 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.578841 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.578842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.578842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.578911 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.579147 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.579148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.579149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.579301 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.579379 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.579501 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.579573 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab.
Entry for principal oryx/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab.
I20260809 16:11:42.600641 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:34213
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab
--principal=oryx/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:42.684175 30309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:42.684386 30309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:42.684408 30309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:42.685487 30309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:42.685523 30309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:42.685542 30309 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:11:42.685557 30309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:42.687052 30309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab
--principal=oryx/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.224.190:34213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:42.687247 30309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:42.687443 30309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:42.692116 30319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.692108 30322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:42.692107 30316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:42.692430 30309 server_base.cc:1034] running on GCE node
I20260809 16:11:42.692634 30309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:42.692857 30309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:42.693985 30309 hybrid_clock.cc:648] HybridClock initialized: now 1786291902693976 us; error 27 us; skew 500 ppm
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:42.696419 30309 init.cc:377] Logged in from keytab as oryx/127.16.224.131@KRBTEST.COM (short username oryx)
I20260809 16:11:42.696856 30309 webserver.cc:460] Webserver started at http://127.16.224.131:40911/ using document root <none> and password file <none>
I20260809 16:11:42.697070 30309 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:42.697114 30309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:42.697237 30309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:42.698403 30309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "364a6d6b39a2b5e1cab3b5b7faa9088b"
server_key_iv: "690e99152616e3a5115134fa3a56a7fb"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.698747 30309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "364a6d6b39a2b5e1cab3b5b7faa9088b"
server_key_iv: "690e99152616e3a5115134fa3a56a7fb"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.700006 30309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:42.700636 30341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.700786 30309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:42.700875 30309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
format_stamp: "Formatted at 2026-08-09 16:11:42 on dist-test-slave-fgb3"
server_key: "364a6d6b39a2b5e1cab3b5b7faa9088b"
server_key_iv: "690e99152616e3a5115134fa3a56a7fb"
server_key_version: "encryptionkey@0"
I20260809 16:11:42.700933 30309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:42.712224 30309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:42.713835 30309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:42.713966 30309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:42.714519 30309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:42.714558 30309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.714586 30309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:42.714601 30309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:42.721199 30309 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:45937
I20260809 16:11:42.721309 30517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:45937 every 8 connection(s)
I20260809 16:11:42.721570 30309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:11:42.724251 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.714904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:60263 (local address 127.16.224.190:34213)
0809 16:11:42.714974 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.714977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.715192 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:11:42.715873 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.715875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.715889 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.715949 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.716777 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.717537 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.719081 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.719379 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.719442 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.721705 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.721706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.721706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.721714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.723075 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.723434 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.723435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.723435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.723484 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.723723 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.723723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.723724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.723819 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.723871 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.724132 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.724189 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:11:42.724445 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.714877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:34213 (local address 127.16.224.131:60263)
0809 16:11:42.715056 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.715103 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.715721 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.716049 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.716054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.716404 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:42.716689 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.716696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.717634 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.717636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.718976 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.718980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.719469 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.719470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.719586 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.720068 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:42.720076 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:42.721599 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:42.723144 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.723150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.723152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.723375 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.723559 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.723560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.723560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.723670 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.723864 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:42.723867 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:42.724007 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.724108 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.724250 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260809 16:11:42.724866 30518 heartbeater.cc:347] Connected to a master server at 127.16.224.190:34213
I20260809 16:11:42.724953 30518 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:42.725133 30518 heartbeater.cc:511] Master 127.16.224.190:34213 requested a full tablet report, sending...
I20260809 16:11:42.725328 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 30309
I20260809 16:11:42.725443 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:42.725554 29687 ts_manager.cc:194] Registered new tserver with Master: 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:42.725656 17282 external_mini_cluster.cc:1428] Setting key 1c60474113889fcbe0999f9dd08322a1
I20260809 16:11:42.726004 29687 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:60263
I20260809 16:11:42.726702 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:11:42.731490 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.726430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:38929 (local address 127.16.224.190:34213)
0809 16:11:42.726463 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.726464 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.726524 (+ 60us) server_negotiation.cc:407] Connection header received
0809 16:11:42.726614 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.726615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.726624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.726665 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.727002 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.727522 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.728263 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.728461 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.728514 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.729355 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.729356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.729357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.729364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.730539 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.730816 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.730817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.730818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.730877 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.731034 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.731035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.731035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.731135 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.731187 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.731330 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.731403 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:11:42.731504 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.726276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:34213 (local address 127.16.224.131:38929)
0809 16:11:42.726461 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.726472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.726550 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.726680 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.726681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.726866 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:42.726928 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.726930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.727607 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.727608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.728189 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.728191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.728520 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.728522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.728590 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.728988 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:42.728993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:42.729276 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:42.730604 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.730605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.730606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.730756 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.730927 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.730928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.730928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.730985 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.731186 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:42.731187 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:42.731241 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.731300 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.731404 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:11:42.747973 29774 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:42.743087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56986 (local address 127.16.224.190:34213)
0809 16:11:42.743126 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.743128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.743138 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:42.743196 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.743196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.743205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.743249 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.743765 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.744240 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.744971 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.745153 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.745210 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.747898 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56986: BlockingRecv error: recv got EOF from 127.0.0.1:56986 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:42.754900 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.748766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57000 (local address 127.16.224.190:34213)
0809 16:11:42.748813 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.748815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.748896 (+ 81us) server_negotiation.cc:407] Connection header received
0809 16:11:42.748955 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.748956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.748965 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.749011 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.749519 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.750014 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.750727 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.750921 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.750973 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.752882 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.752883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.752883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.752890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.753902 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.754186 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.754187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.754187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.754236 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.754436 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.754437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.754437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.754538 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.754621 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.754743 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.754808 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:11:42.756243 29687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:42.757686 29687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:42.763775 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.759748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:44411 (local address 127.0.0.1:47812)
0809 16:11:42.760259 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.760267 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.760312 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.760637 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.760638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.760639 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.760683 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.760684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.761843 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.761844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.762648 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.762651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.763468 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.763469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.763559 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.763574 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.763650 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.763699 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260809 16:11:42.763803 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.759964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.16.224.130:44411)
0809 16:11:42.760137 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.760141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.760356 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:11:42.760423 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.760426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.760442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.760520 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.760767 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.761769 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.762726 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.763381 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.763477 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.763536 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.763670 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.763711 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260809 16:11:42.764133 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.759740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44729 (local address 127.0.0.1:43618)
0809 16:11:42.760172 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.760180 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.760214 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.760514 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.760515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.760517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.760567 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.760569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.761536 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.761537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.762915 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.762917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.763923 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.763924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.763977 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.763980 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.764026 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.764064 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":41,"thread_start_us":56,"threads_started":1}
I20260809 16:11:42.764189 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.759959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43618 (local address 127.16.224.129:44729)
0809 16:11:42.760383 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.760387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.760396 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:42.760430 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.760431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.760445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.760520 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.760622 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.761455 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.763010 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.763756 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.763835 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.763893 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.764058 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.764099 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":87,"threads_started":1}
I20260809 16:11:42.764464 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.759972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54040 (local address 127.16.224.131:45937)
0809 16:11:42.760135 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.760139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.760149 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:42.760185 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.760186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.760199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.760265 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.760454 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.761547 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.762995 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.763680 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.763762 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.763818 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.764315 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.764370 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260809 16:11:42.764676 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.759740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:45937 (local address 127.0.0.1:54040)
0809 16:11:42.759897 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.759940 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.760003 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.760294 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.760298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.760300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.760377 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.760378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.761627 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.761627 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.762907 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.762910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.764065 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.764066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.764130 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.764134 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.764566 (+ 432us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.764612 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":41,"thread_start_us":48,"threads_started":1}
I20260809 16:11:42.764690 30188 tablet_service.cc:1467] Processing CreateTablet for tablet e0c6341ee0d841b785793a5b0bd9bbb4 (DEFAULT_TABLE table=test-table [id=7bc3cd4b217c4c61a6117ee4735b1698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:42.764787 29936 tablet_service.cc:1467] Processing CreateTablet for tablet e0c6341ee0d841b785793a5b0bd9bbb4 (DEFAULT_TABLE table=test-table [id=7bc3cd4b217c4c61a6117ee4735b1698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:42.765110 30188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0c6341ee0d841b785793a5b0bd9bbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.765136 29936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0c6341ee0d841b785793a5b0bd9bbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.765332 30423 tablet_service.cc:1467] Processing CreateTablet for tablet e0c6341ee0d841b785793a5b0bd9bbb4 (DEFAULT_TABLE table=test-table [id=7bc3cd4b217c4c61a6117ee4735b1698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:42.765631 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0c6341ee0d841b785793a5b0bd9bbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.768544 30555 tablet_bootstrap.cc:492] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Bootstrap starting.
I20260809 16:11:42.768890 30556 tablet_bootstrap.cc:492] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Bootstrap starting.
I20260809 16:11:42.769423 30555 tablet_bootstrap.cc:654] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.769481 30556 tablet_bootstrap.cc:654] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.769764 30555 log.cc:824] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:42.769774 30556 log.cc:824] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:42.770116 30560 tablet_bootstrap.cc:492] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Bootstrap starting.
I20260809 16:11:42.770562 30555 tablet_bootstrap.cc:492] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: No bootstrap required, opened a new log
I20260809 16:11:42.770588 30556 tablet_bootstrap.cc:492] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: No bootstrap required, opened a new log
I20260809 16:11:42.770619 30555 ts_tablet_manager.cc:1397] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:42.770645 30556 ts_tablet_manager.cc:1397] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:42.772089 30555 raft_consensus.cc:348] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.772222 30555 raft_consensus.cc:374] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.772243 30555 raft_consensus.cc:729] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.772154 30556 raft_consensus.cc:348] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.772294 30556 raft_consensus.cc:374] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.772336 30556 raft_consensus.cc:729] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0096f42e04394942012c0177703d7, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.772372 30555 consensus_queue.cc:260] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.772439 30556 consensus_queue.cc:260] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.772644 30555 ts_tablet_manager.cc:1428] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:42.772693 30296 heartbeater.cc:503] Master 127.16.224.190:34213 was elected leader, sending a full tablet report...
I20260809 16:11:42.772751 30050 heartbeater.cc:503] Master 127.16.224.190:34213 was elected leader, sending a full tablet report...
I20260809 16:11:42.773002 30556 ts_tablet_manager.cc:1428] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:11:42.773617 30560 tablet_bootstrap.cc:654] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.773945 30560 log.cc:824] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:42.774598 30560 tablet_bootstrap.cc:492] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: No bootstrap required, opened a new log
I20260809 16:11:42.774652 30560 ts_tablet_manager.cc:1397] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 16:11:42.776135 30560 raft_consensus.cc:348] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.776254 30560 raft_consensus.cc:374] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.776275 30560 raft_consensus.cc:729] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3e2a385506448f9cfb6c1ef1b74184, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.776377 30560 consensus_queue.cc:260] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.776602 30560 ts_tablet_manager.cc:1428] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:11:42.776648 30518 heartbeater.cc:503] Master 127.16.224.190:34213 was elected leader, sending a full tablet report...
W20260809 16:11:42.820708 30298 tablet.cc:2369] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:42.841887 30568 raft_consensus.cc:482] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:42.841886 30570 raft_consensus.cc:482] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:42.842025 30570 raft_consensus.cc:504] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.842036 30568 raft_consensus.cc:504] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.842432 30568 leader_election.cc:288] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:42.842435 30570 leader_election.cc:288] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411)
I20260809 16:11:42.845888 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:44411 (local address 127.16.224.129:41185)
0809 16:11:42.842753 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.842763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.842806 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.843046 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.843047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.843048 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.843121 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.843124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.844036 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.844036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.844871 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.844874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.845634 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.845635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.845724 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.845728 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.845770 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.845811 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 16:11:42.846280 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:44411 (local address 127.16.224.131:47601)
0809 16:11:42.842688 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.842700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.842742 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.843207 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.843208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.843210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.843259 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.843261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.844108 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.844109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.845283 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.845287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.846057 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.846058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.846135 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.846139 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.846166 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.846209 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":23}
I20260809 16:11:42.846366 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:41185 (local address 127.16.224.130:44411)
0809 16:11:42.842893 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.842895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.842902 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:42.842933 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.842933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.842943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.842991 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.843189 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.843942 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.844969 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.845524 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.845593 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.845602 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.846234 (+ 632us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.846274 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:11:42.846611 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:47183 (local address 127.16.224.131:45937)
0809 16:11:42.842853 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.842855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.843020 (+ 165us) server_negotiation.cc:407] Connection header received
0809 16:11:42.843096 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.843097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.843104 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.843127 (+ 23us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.843355 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.844047 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.845718 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.846198 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.846232 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.846239 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.846522 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.846557 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:11:42.846647 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:47601 (local address 127.16.224.130:44411)
0809 16:11:42.843074 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.843076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.843082 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:11:42.843102 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.843102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.843110 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.843314 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.843342 (+ 28us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.844026 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.845385 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.845972 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.846491 (+ 519us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.846501 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.846519 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.846562 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260809 16:11:42.846679 30229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0946bd92b7984c2b97cfa60d287fec02" is_pre_election: true
I20260809 16:11:42.846848 30229 raft_consensus.cc:2454] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa0096f42e04394942012c0177703d7 in term 0.
I20260809 16:11:42.846858 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:45937 (local address 127.16.224.129:47183)
0809 16:11:42.842969 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.842978 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.843039 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.843184 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.843185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.843186 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.843253 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.843254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.844135 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.844136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.845631 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.845634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.846349 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.846350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.846442 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.846446 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.846728 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.846772 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260809 16:11:42.846885 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44729 (local address 127.16.224.131:40903)
0809 16:11:42.842830 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.842839 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.842875 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.843139 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.843140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.843141 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.843206 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.843209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.844135 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.844136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.845922 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.845925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.846654 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.846655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.846731 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.846735 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.846780 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.846816 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":107,"threads_started":1}
I20260809 16:11:42.847008 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0946bd92b7984c2b97cfa60d287fec02" is_pre_election: true
I20260809 16:11:42.847085 30225 raft_consensus.cc:2454] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d3e2a385506448f9cfb6c1ef1b74184 in term 0.
I20260809 16:11:42.847100 29840 leader_election.cc:302] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7; no voters:
I20260809 16:11:42.847103 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.842886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:40903 (local address 127.16.224.129:44729)
0809 16:11:42.842921 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.842923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.842931 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:42.842954 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.842955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.842962 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.843011 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.843288 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.844047 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.846016 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.846526 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.846572 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.846579 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.847012 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.847051 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":19}
I20260809 16:11:42.847209 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" is_pre_election: true
I20260809 16:11:42.847289 30568 raft_consensus.cc:2781] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:42.847307 30454 raft_consensus.cc:2454] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa0096f42e04394942012c0177703d7 in term 0.
I20260809 16:11:42.847282 30343 leader_election.cc:302] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 3d3e2a385506448f9cfb6c1ef1b74184; no voters:
I20260809 16:11:42.847333 30568 raft_consensus.cc:482] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:42.847354 30568 raft_consensus.cc:3037] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:42.847491 30570 raft_consensus.cc:2781] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:42.847538 30570 raft_consensus.cc:482] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:42.847572 30570 raft_consensus.cc:3037] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:42.848042 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7" is_pre_election: true
I20260809 16:11:42.848181 30568 raft_consensus.cc:504] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.848308 29967 raft_consensus.cc:2379] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d3e2a385506448f9cfb6c1ef1b74184 in current term 1: Already voted for candidate 7fa0096f42e04394942012c0177703d7 in this term.
I20260809 16:11:42.848469 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0946bd92b7984c2b97cfa60d287fec02"
I20260809 16:11:42.848554 30225 raft_consensus.cc:3037] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:42.848578 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
I20260809 16:11:42.848722 30570 raft_consensus.cc:504] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.848771 30568 leader_election.cc:288] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 election: Requested vote from peers 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:42.848829 30454 raft_consensus.cc:2379] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fa0096f42e04394942012c0177703d7 in current term 1: Already voted for candidate 3d3e2a385506448f9cfb6c1ef1b74184 in this term.
I20260809 16:11:42.848869 30570 leader_election.cc:288] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [CANDIDATE]: Term 1 election: Requested vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411)
I20260809 16:11:42.849000 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7"
I20260809 16:11:42.849064 29967 raft_consensus.cc:2379] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d3e2a385506448f9cfb6c1ef1b74184 in current term 1: Already voted for candidate 7fa0096f42e04394942012c0177703d7 in this term.
I20260809 16:11:42.849056 30229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0c6341ee0d841b785793a5b0bd9bbb4" candidate_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0946bd92b7984c2b97cfa60d287fec02"
I20260809 16:11:42.849308 30343 leader_election.cc:302] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d3e2a385506448f9cfb6c1ef1b74184; no voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7
I20260809 16:11:42.849398 30570 raft_consensus.cc:2726] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 16:11:42.849495 30225 raft_consensus.cc:2454] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fa0096f42e04394942012c0177703d7 in term 1.
I20260809 16:11:42.849704 29840 leader_election.cc:302] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7; no voters: 3d3e2a385506448f9cfb6c1ef1b74184
I20260809 16:11:42.849803 30568 raft_consensus.cc:2781] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:42.849960 30568 raft_consensus.cc:686] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0096f42e04394942012c0177703d7, State: Running, Role: LEADER
I20260809 16:11:42.850046 30568 consensus_queue.cc:237] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.850875 29687 catalog_manager.cc:5495] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0096f42e04394942012c0177703d7 (127.16.224.129). New cstate: current_term: 1 leader_uuid: "7fa0096f42e04394942012c0177703d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:42.878720 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.876407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43628 (local address 127.16.224.129:44729)
0809 16:11:42.876477 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.876479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.876503 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:11:42.876544 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.876545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.876565 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.876622 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:42.876746 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.877231 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.877976 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.878172 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.878227 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.878404 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:42.878514 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:42.878585 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.878623 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.878643 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:11:42.880470 30454 raft_consensus.cc:1261] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Refusing update from remote peer 7fa0096f42e04394942012c0177703d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:42.880470 30225 raft_consensus.cc:1261] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Refusing update from remote peer 7fa0096f42e04394942012c0177703d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:42.880723 30576 consensus_queue.cc:1035] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:42.880859 30568 consensus_queue.cc:1035] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:42.883109 30580 mvcc.cc:205] Tried to move back new op lower bound from 7316651634195988480 to 7316651634074349568. Current Snapshot: MvccSnapshot[applied={T|T < 7316651634195988480}]
I20260809 16:11:42.884029 30584 mvcc.cc:205] Tried to move back new op lower bound from 7316651634195988480 to 7316651634074349568. Current Snapshot: MvccSnapshot[applied={T|T < 7316651634195988480}]
I20260809 16:11:42.885252 29687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:42.885349 29687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:42.886086 29687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Sending DeleteTablet for 3 replicas of tablet e0c6341ee0d841b785793a5b0bd9bbb4
I20260809 16:11:42.886464 30423 tablet_service.cc:1514] Processing DeleteTablet for tablet e0c6341ee0d841b785793a5b0bd9bbb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:42 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:54040
I20260809 16:11:42.886439 29936 tablet_service.cc:1514] Processing DeleteTablet for tablet e0c6341ee0d841b785793a5b0bd9bbb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:42 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:43618
I20260809 16:11:42.886696 30596 tablet_replica.cc:330] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: stopping tablet replica
I20260809 16:11:42.886801 30596 raft_consensus.cc:2229] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:42.886832 30188 tablet_service.cc:1514] Processing DeleteTablet for tablet e0c6341ee0d841b785793a5b0bd9bbb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:42 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:47812
I20260809 16:11:42.886893 30596 raft_consensus.cc:2258] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:42.886832 29687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:42.886970 29687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:42.887142 30597 tablet_replica.cc:330] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: stopping tablet replica
I20260809 16:11:42.887249 30597 raft_consensus.cc:2229] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:42.887352 30596 ts_tablet_manager.cc:1905] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:42.887380 30597 raft_consensus.cc:2258] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:42.887818 30597 ts_tablet_manager.cc:1905] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:42.888586 30188 tablet_service.cc:1467] Processing CreateTablet for tablet b68ecb970ed34e15b463d5cc6cc1e971 (DEFAULT_TABLE table=test-table [id=fad66fd2241044cfb071cd54bd6c6e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:42.888710 30188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68ecb970ed34e15b463d5cc6cc1e971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.888981 30594 tablet_replica.cc:330] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: stopping tablet replica
I20260809 16:11:42.888991 30423 tablet_service.cc:1467] Processing CreateTablet for tablet b68ecb970ed34e15b463d5cc6cc1e971 (DEFAULT_TABLE table=test-table [id=fad66fd2241044cfb071cd54bd6c6e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:42.889077 30594 raft_consensus.cc:2229] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:42.889086 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68ecb970ed34e15b463d5cc6cc1e971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.889150 30594 raft_consensus.cc:2258] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:42.889599 30594 ts_tablet_manager.cc:1905] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:42.889961 30555 tablet_bootstrap.cc:492] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: Bootstrap starting.
I20260809 16:11:42.890231 30560 tablet_bootstrap.cc:492] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: Bootstrap starting.
I20260809 16:11:42.890417 30555 tablet_bootstrap.cc:654] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.890436 29936 tablet_service.cc:1467] Processing CreateTablet for tablet b68ecb970ed34e15b463d5cc6cc1e971 (DEFAULT_TABLE table=test-table [id=fad66fd2241044cfb071cd54bd6c6e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:42.890550 29936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68ecb970ed34e15b463d5cc6cc1e971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.890637 30560 tablet_bootstrap.cc:654] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.891010 30596 ts_tablet_manager.cc:1918] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:42.891083 30596 log.cc:1196] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/e0c6341ee0d841b785793a5b0bd9bbb4
I20260809 16:11:42.891333 30596 ts_tablet_manager.cc:1939] T e0c6341ee0d841b785793a5b0bd9bbb4 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting consensus metadata
I20260809 16:11:42.891728 29664 catalog_manager.cc:4841] TS 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937): tablet e0c6341ee0d841b785793a5b0bd9bbb4 (table test-table [id=7bc3cd4b217c4c61a6117ee4735b1698]) successfully deleted
I20260809 16:11:42.891825 30556 tablet_bootstrap.cc:492] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Bootstrap starting.
I20260809 16:11:42.892218 30556 tablet_bootstrap.cc:654] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.892586 30597 ts_tablet_manager.cc:1918] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:42.892694 30597 log.cc:1196] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/e0c6341ee0d841b785793a5b0bd9bbb4
I20260809 16:11:42.892760 30556 tablet_bootstrap.cc:492] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: No bootstrap required, opened a new log
I20260809 16:11:42.892807 30556 ts_tablet_manager.cc:1397] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:42.892993 30594 ts_tablet_manager.cc:1918] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:42.892953 30556 raft_consensus.cc:348] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.893066 30556 raft_consensus.cc:374] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.893100 30556 raft_consensus.cc:729] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0096f42e04394942012c0177703d7, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.893150 30556 consensus_queue.cc:260] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.893187 30597 ts_tablet_manager.cc:1939] T e0c6341ee0d841b785793a5b0bd9bbb4 P 0946bd92b7984c2b97cfa60d287fec02: Deleting consensus metadata
I20260809 16:11:42.893260 30556 ts_tablet_manager.cc:1428] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:42.893483 30594 log.cc:1196] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/e0c6341ee0d841b785793a5b0bd9bbb4
I20260809 16:11:42.893573 29665 catalog_manager.cc:4841] TS 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411): tablet e0c6341ee0d841b785793a5b0bd9bbb4 (table test-table [id=7bc3cd4b217c4c61a6117ee4735b1698]) successfully deleted
I20260809 16:11:42.893729 30594 ts_tablet_manager.cc:1939] T e0c6341ee0d841b785793a5b0bd9bbb4 P 7fa0096f42e04394942012c0177703d7: Deleting consensus metadata
I20260809 16:11:42.894083 29668 catalog_manager.cc:4841] TS 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729): tablet e0c6341ee0d841b785793a5b0bd9bbb4 (table test-table [id=7bc3cd4b217c4c61a6117ee4735b1698]) successfully deleted
I20260809 16:11:42.894213 30560 tablet_bootstrap.cc:492] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: No bootstrap required, opened a new log
I20260809 16:11:42.894251 30560 ts_tablet_manager.cc:1397] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 16:11:42.894342 30555 tablet_bootstrap.cc:492] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: No bootstrap required, opened a new log
I20260809 16:11:42.894387 30555 ts_tablet_manager.cc:1397] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 16:11:42.894399 30560 raft_consensus.cc:348] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.894471 30560 raft_consensus.cc:374] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.894503 30560 raft_consensus.cc:729] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3e2a385506448f9cfb6c1ef1b74184, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.894513 30555 raft_consensus.cc:348] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.894546 30560 consensus_queue.cc:260] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.894577 30555 raft_consensus.cc:374] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.894598 30555 raft_consensus.cc:729] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.894626 30560 ts_tablet_manager.cc:1428] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.894634 30555 consensus_queue.cc:260] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.894716 30555 ts_tablet_manager.cc:1428] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.908039 30566 raft_consensus.cc:482] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:42.908133 30566 raft_consensus.cc:504] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.908470 30566 leader_election.cc:288] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:42.912001 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.908720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:40445 (local address 127.16.224.129:44729)
0809 16:11:42.908763 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.908765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.908994 (+ 229us) server_negotiation.cc:407] Connection header received
0809 16:11:42.909041 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.909042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.909051 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.909100 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.909234 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.909700 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.911136 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.911692 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.911770 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.911780 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.911898 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.911940 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:11:42.912016 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.908613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44729 (local address 127.16.224.130:40445)
0809 16:11:42.908915 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.908925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.908992 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.909102 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.909102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.909104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.909166 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.909167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.909769 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.909770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.911048 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.911051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.911767 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.911767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.911828 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.911832 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.911877 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.911918 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":168,"thread_start_us":86,"threads_started":1}
I20260809 16:11:42.912359 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68ecb970ed34e15b463d5cc6cc1e971" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7" is_pre_election: true
I20260809 16:11:42.912452 29967 raft_consensus.cc:2454] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 0.
I20260809 16:11:42.912634 30102 leader_election.cc:302] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7; no voters:
I20260809 16:11:42.912783 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.908611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:45937 (local address 127.16.224.130:41229)
0809 16:11:42.908668 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.908679 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.908720 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.908917 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.908919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.908921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:11:42.908987 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.908989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.909996 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.909997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.911048 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.911051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.912519 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.912520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.912619 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.912623 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.912676 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.912712 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260809 16:11:42.912803 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.908744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:41229 (local address 127.16.224.131:45937)
0809 16:11:42.908791 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.908793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.908802 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:42.908829 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.908830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.908839 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.908896 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:11:42.909143 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.909899 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.911818 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.912405 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.912474 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.912483 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.912699 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.912744 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:11:42.912814 30566 raft_consensus.cc:2781] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:42.912858 30566 raft_consensus.cc:482] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:42.912881 30566 raft_consensus.cc:3037] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:42.913100 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68ecb970ed34e15b463d5cc6cc1e971" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" is_pre_election: true
I20260809 16:11:42.913175 30454 raft_consensus.cc:2454] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 0.
I20260809 16:11:42.913707 30566 raft_consensus.cc:504] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.913818 30566 leader_election.cc:288] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 election: Requested vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:42.913976 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68ecb970ed34e15b463d5cc6cc1e971" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
I20260809 16:11:42.914047 30454 raft_consensus.cc:3037] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:42.914752 30454 raft_consensus.cc:2454] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 1.
I20260809 16:11:42.914871 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68ecb970ed34e15b463d5cc6cc1e971" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7"
I20260809 16:11:42.914947 29967 raft_consensus.cc:3037] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:42.914927 30094 leader_election.cc:302] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 3d3e2a385506448f9cfb6c1ef1b74184; no voters:
I20260809 16:11:42.915011 30566 raft_consensus.cc:2781] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:42.915143 30566 raft_consensus.cc:686] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 LEADER]: Becoming Leader. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Running, Role: LEADER
I20260809 16:11:42.915212 30566 consensus_queue.cc:237] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.915722 29967 raft_consensus.cc:2454] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 1.
I20260809 16:11:42.916179 29688 catalog_manager.cc:5495] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } health_report { overall_health: UNKNOWN } } }
W20260809 16:11:42.925767 30051 tablet.cc:2369] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 16:11:42 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291902, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.190@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:42.946113 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.939000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.16.224.190:34213)
0809 16:11:42.939069 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:42.939071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:42.939184 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:11:42.939253 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:42.939254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:42.939263 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:42.939310 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:42.939880 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.940357 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.941356 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.941531 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.941572 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.943798 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:42.943799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:42.943799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:42.943805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:42.945012 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.945447 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.945448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.945448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.945497 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:42.945682 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:42.945682 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:42.945683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:42.945773 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:42.945816 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:42.945985 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 16:11:42.946049 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:11:42.946281 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:42.938929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:34213 (local address 127.0.0.1:57004)
0809 16:11:42.939112 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:42.939123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:42.939196 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:42.939330 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:42.939331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:42.939751 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:42.939809 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.939812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.940437 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.940438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.941285 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:42.941288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:42.941588 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:42.941589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:42.941647 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:42.942146 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:42.942151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:42.943717 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:42.945093 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.945102 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.945104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.945377 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.945551 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:42.945552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:42.945553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:42.945628 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:42.945830 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:42.945837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:42.945892 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:11:42.945969 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:11:42.946126 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":36,"thread_start_us":96,"threads_started":1}
I20260809 16:11:42.948359 29688 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:57004:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:11:42.948567 29688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:42.950021 29936 tablet_service.cc:1467] Processing CreateTablet for tablet 9d859613dce04e00be38b028d64439c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=12cfce57082f471aa1e98899a6c72630]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:42.950011 30423 tablet_service.cc:1467] Processing CreateTablet for tablet 9d859613dce04e00be38b028d64439c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=12cfce57082f471aa1e98899a6c72630]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:42.950174 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d859613dce04e00be38b028d64439c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.950174 29936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d859613dce04e00be38b028d64439c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.950487 30188 tablet_service.cc:1467] Processing CreateTablet for tablet 9d859613dce04e00be38b028d64439c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=12cfce57082f471aa1e98899a6c72630]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:11:42.950636 30188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d859613dce04e00be38b028d64439c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:42.951627 30560 tablet_bootstrap.cc:492] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: Bootstrap starting.
I20260809 16:11:42.951627 30556 tablet_bootstrap.cc:492] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: Bootstrap starting.
I20260809 16:11:42.951951 30556 tablet_bootstrap.cc:654] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.952004 30560 tablet_bootstrap.cc:654] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.952299 30555 tablet_bootstrap.cc:492] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: Bootstrap starting.
I20260809 16:11:42.952898 30560 tablet_bootstrap.cc:492] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: No bootstrap required, opened a new log
I20260809 16:11:42.952951 30560 ts_tablet_manager.cc:1397] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:42.952997 30555 tablet_bootstrap.cc:654] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:42.953095 30560 raft_consensus.cc:348] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.953163 30560 raft_consensus.cc:374] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.953181 30560 raft_consensus.cc:729] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3e2a385506448f9cfb6c1ef1b74184, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.953226 30560 consensus_queue.cc:260] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.953307 30560 ts_tablet_manager.cc:1428] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.953356 30570 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:42.953456 30570 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:42.953701 30556 tablet_bootstrap.cc:492] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: No bootstrap required, opened a new log
I20260809 16:11:42.953754 30556 ts_tablet_manager.cc:1397] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:11:42.953892 30556 raft_consensus.cc:348] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.953948 30556 raft_consensus.cc:374] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.953964 30556 raft_consensus.cc:729] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0096f42e04394942012c0177703d7, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.954027 30556 consensus_queue.cc:260] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.954123 30556 ts_tablet_manager.cc:1428] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.954164 30576 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:42.954267 30576 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:42.954546 30555 tablet_bootstrap.cc:492] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: No bootstrap required, opened a new log
I20260809 16:11:42.954600 30555 ts_tablet_manager.cc:1397] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:11:42.954732 30555 raft_consensus.cc:348] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.954804 30555 raft_consensus.cc:374] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:42.954823 30555 raft_consensus.cc:729] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Initialized, Role: FOLLOWER
I20260809 16:11:42.954876 30555 consensus_queue.cc:260] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:42.954964 30555 ts_tablet_manager.cc:1428] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:42.955027 30566 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:42.955147 30566 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 16:11:42.972306 30520 tablet.cc:2369] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:11:43.203678 30566 raft_consensus.cc:482] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:43.203797 30566 raft_consensus.cc:504] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.203984 30566 leader_election.cc:288] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:43.204243 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d859613dce04e00be38b028d64439c6" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" is_pre_election: true
I20260809 16:11:43.204245 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d859613dce04e00be38b028d64439c6" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7" is_pre_election: true
I20260809 16:11:43.204350 29967 raft_consensus.cc:2454] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 0.
I20260809 16:11:43.204348 30454 raft_consensus.cc:2454] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 0.
I20260809 16:11:43.204509 30094 leader_election.cc:302] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 3d3e2a385506448f9cfb6c1ef1b74184; no voters:
I20260809 16:11:43.204618 30566 raft_consensus.cc:2781] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:43.204659 30566 raft_consensus.cc:482] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:43.204685 30566 raft_consensus.cc:3037] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.205355 30566 raft_consensus.cc:504] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.205475 30566 leader_election.cc:288] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 election: Requested vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:43.205664 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d859613dce04e00be38b028d64439c6" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
I20260809 16:11:43.205660 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d859613dce04e00be38b028d64439c6" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7"
I20260809 16:11:43.205734 29967 raft_consensus.cc:3037] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.205734 30454 raft_consensus.cc:3037] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.206362 30454 raft_consensus.cc:2454] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 1.
I20260809 16:11:43.206374 29967 raft_consensus.cc:2454] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 1.
I20260809 16:11:43.206524 30102 leader_election.cc:302] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7; no voters:
I20260809 16:11:43.206633 30566 raft_consensus.cc:2781] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:43.206676 30566 raft_consensus.cc:686] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 LEADER]: Becoming Leader. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Running, Role: LEADER
I20260809 16:11:43.206714 30566 consensus_queue.cc:237] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.207008 30606 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: TxnStatusTablet state changed. Reason: New leader 0946bd92b7984c2b97cfa60d287fec02. Latest consensus state: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:43.207099 30606 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:43.207285 30622 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:11:43.207414 30622 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:11:43.207379 29687 catalog_manager.cc:5495] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:43.238746 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.235976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:44411 (local address 127.0.0.1:47822)
0809 16:11:43.236068 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:43.236080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:43.236138 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:43.236313 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:43.236314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:43.236315 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:43.236365 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.236368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.237222 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.237223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.237778 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.237779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.238249 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.238250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.238292 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.238305 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.238621 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.238625 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:11:43.238649 (+ 24us) client_negotiation.cc:240] Negotiation successful
0809 16:11:43.238685 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 16:11:43.238760 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.236033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.16.224.130:44411)
0809 16:11:43.236088 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.236090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.236119 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:11:43.236218 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.236219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.236242 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.236299 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:43.236404 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.237153 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.237846 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.238185 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.238255 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.238413 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.238565 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.238640 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.238658 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.238681 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:11:43.239841 30454 raft_consensus.cc:1261] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Refusing update from remote peer 0946bd92b7984c2b97cfa60d287fec02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.239930 29967 raft_consensus.cc:1261] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Refusing update from remote peer 0946bd92b7984c2b97cfa60d287fec02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.240202 30566 consensus_queue.cc:1035] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.240320 30606 consensus_queue.cc:1035] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.240835 30570 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: TxnStatusTablet state changed. Reason: New leader 0946bd92b7984c2b97cfa60d287fec02. Latest consensus state: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:43.240921 30570 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:43.240953 30576 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: TxnStatusTablet state changed. Reason: New leader 0946bd92b7984c2b97cfa60d287fec02. Latest consensus state: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:43.241034 30576 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:43.242221 30566 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:43.242301 30566 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:43.242872 30606 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:43.242947 30606 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 0946bd92b7984c2b97cfa60d287fec02: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:11:43.242959 30576 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:43.243032 30576 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 7fa0096f42e04394942012c0177703d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:43.243568 30570 tablet_replica.cc:439] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } } }
I20260809 16:11:43.243654 30570 tablet_replica.cc:442] T 9d859613dce04e00be38b028d64439c6 P 3d3e2a385506448f9cfb6c1ef1b74184: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:11:43.243697 30582 mvcc.cc:205] Tried to move back new op lower bound from 7316651635668508672 to 7316651635535130624. Current Snapshot: MvccSnapshot[applied={T|T < 7316651635668508672 or (T in {7316651635668508672})}]
I20260809 16:11:43.246291 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.243985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47832 (local address 127.16.224.130:44411)
0809 16:11:43.244023 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.244025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.244032 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:43.244050 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.244051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.244060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.244105 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:43.244264 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.244964 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.245656 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.245952 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.246027 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.246100 (+ 73us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.246132 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.246176 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.246209 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.246227 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:11:43.251500 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.249014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:54697 (local address 127.16.224.130:44411)
0809 16:11:43.249045 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.249046 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.249080 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:11:43.249125 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.249126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.249134 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.249192 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:43.249282 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.250031 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.250790 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.251089 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.251155 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.251278 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.251308 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.251348 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.251391 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.251409 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:11:43.251503 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.248856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:44411 (local address 127.16.224.130:54697)
0809 16:11:43.249027 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:43.249036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:43.249088 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:43.249196 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:43.249197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:43.249198 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:43.249248 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.249249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.250115 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.250116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.250719 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.250721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.251165 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.251166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.251206 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.251218 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.251357 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.251358 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:11:43.251388 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 16:11:43.251419 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260809 16:11:43.253592 30454 raft_consensus.cc:1261] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Refusing update from remote peer 0946bd92b7984c2b97cfa60d287fec02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.253630 29967 raft_consensus.cc:1261] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Refusing update from remote peer 0946bd92b7984c2b97cfa60d287fec02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.253788 30636 consensus_queue.cc:1035] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.253892 30606 consensus_queue.cc:1035] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.264941 29686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:43.265043 29686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:43.265631 29686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Sending DeleteTablet for 3 replicas of tablet b68ecb970ed34e15b463d5cc6cc1e971
I20260809 16:11:43.265846 29936 tablet_service.cc:1514] Processing DeleteTablet for tablet b68ecb970ed34e15b463d5cc6cc1e971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:43618
I20260809 16:11:43.265923 30187 tablet_service.cc:1514] Processing DeleteTablet for tablet b68ecb970ed34e15b463d5cc6cc1e971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:47812
I20260809 16:11:43.265928 30423 tablet_service.cc:1514] Processing DeleteTablet for tablet b68ecb970ed34e15b463d5cc6cc1e971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:54040
I20260809 16:11:43.266016 30594 tablet_replica.cc:330] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: stopping tablet replica
I20260809 16:11:43.266075 30594 raft_consensus.cc:2229] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:43.266114 30594 raft_consensus.cc:2258] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.266181 30597 tablet_replica.cc:330] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: stopping tablet replica
I20260809 16:11:43.266232 30597 raft_consensus.cc:2229] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:43.266341 30597 raft_consensus.cc:2258] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.266431 30596 tablet_replica.cc:330] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: stopping tablet replica
I20260809 16:11:43.266461 30594 ts_tablet_manager.cc:1905] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.266489 30596 raft_consensus.cc:2229] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:43.266522 30596 raft_consensus.cc:2258] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.266944 30597 ts_tablet_manager.cc:1905] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.268149 30594 ts_tablet_manager.cc:1918] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:43.268208 30594 log.cc:1196] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/b68ecb970ed34e15b463d5cc6cc1e971
I20260809 16:11:43.268404 30594 ts_tablet_manager.cc:1939] T b68ecb970ed34e15b463d5cc6cc1e971 P 7fa0096f42e04394942012c0177703d7: Deleting consensus metadata
I20260809 16:11:43.268426 30597 ts_tablet_manager.cc:1918] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:43.268474 30597 log.cc:1196] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/b68ecb970ed34e15b463d5cc6cc1e971
I20260809 16:11:43.268653 30597 ts_tablet_manager.cc:1939] T b68ecb970ed34e15b463d5cc6cc1e971 P 0946bd92b7984c2b97cfa60d287fec02: Deleting consensus metadata
I20260809 16:11:43.268924 29665 catalog_manager.cc:4841] TS 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411): tablet b68ecb970ed34e15b463d5cc6cc1e971 (table test-table [id=fad66fd2241044cfb071cd54bd6c6e8a]) successfully deleted
I20260809 16:11:43.270269 30596 ts_tablet_manager.cc:1905] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.270536 29668 catalog_manager.cc:4841] TS 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729): tablet b68ecb970ed34e15b463d5cc6cc1e971 (table test-table [id=fad66fd2241044cfb071cd54bd6c6e8a]) successfully deleted
I20260809 16:11:43.271742 30596 ts_tablet_manager.cc:1918] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:43.271800 30596 log.cc:1196] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/b68ecb970ed34e15b463d5cc6cc1e971
I20260809 16:11:43.271981 30596 ts_tablet_manager.cc:1939] T b68ecb970ed34e15b463d5cc6cc1e971 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting consensus metadata
I20260809 16:11:43.272276 29664 catalog_manager.cc:4841] TS 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937): tablet b68ecb970ed34e15b463d5cc6cc1e971 (table test-table [id=fad66fd2241044cfb071cd54bd6c6e8a]) successfully deleted
Aug 09 16:11:43 dist-test-slave-fgb3 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:11:43 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:43 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:11:43.290665 29774 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:43.284860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.16.224.190:34213)
0809 16:11:43.284927 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.284930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.285045 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:11:43.285125 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.285126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.285135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.285181 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:43.285881 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.286400 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.287446 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.287617 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.287657 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.290604 (+ 2947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57016: BlockingRecv error: recv got EOF from 127.0.0.1:57016 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 16:11:43 dist-test-slave-fgb3 krb5kdc[29612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM
I20260809 16:11:43.298190 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.291357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.16.224.190:34213)
0809 16:11:43.291409 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.291411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.291526 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:11:43.291556 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.291557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.291568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.291629 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:43.292032 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.292727 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.293358 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.293646 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.293720 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.295592 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:43.295594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:43.295594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:43.295603 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:43.297176 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:43.297508 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:43.297509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:43.297510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:43.297569 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:43.297723 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:43.297724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:43.297724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:43.297830 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:43.297914 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.298037 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.298105 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:11:43.299613 29686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:43.299772 29686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:43.301153 30187 tablet_service.cc:1467] Processing CreateTablet for tablet 711d6e13bd7a49a78a37c8062a5a7d90 (DEFAULT_TABLE table=test-table [id=28b7bfdab0ea41bcac1e008f529a6705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:43.301172 30423 tablet_service.cc:1467] Processing CreateTablet for tablet 711d6e13bd7a49a78a37c8062a5a7d90 (DEFAULT_TABLE table=test-table [id=28b7bfdab0ea41bcac1e008f529a6705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:43.301291 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711d6e13bd7a49a78a37c8062a5a7d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:43.301292 30187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711d6e13bd7a49a78a37c8062a5a7d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:43.302685 30560 tablet_bootstrap.cc:492] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: Bootstrap starting.
I20260809 16:11:43.302809 29936 tablet_service.cc:1467] Processing CreateTablet for tablet 711d6e13bd7a49a78a37c8062a5a7d90 (DEFAULT_TABLE table=test-table [id=28b7bfdab0ea41bcac1e008f529a6705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:43.302925 29936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711d6e13bd7a49a78a37c8062a5a7d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:43.303197 30560 tablet_bootstrap.cc:654] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:43.303570 30555 tablet_bootstrap.cc:492] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: Bootstrap starting.
I20260809 16:11:43.303966 30555 tablet_bootstrap.cc:654] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:43.304015 30556 tablet_bootstrap.cc:492] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: Bootstrap starting.
I20260809 16:11:43.304344 30556 tablet_bootstrap.cc:654] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:43.304646 30560 tablet_bootstrap.cc:492] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: No bootstrap required, opened a new log
I20260809 16:11:43.304697 30560 ts_tablet_manager.cc:1397] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:43.304841 30560 raft_consensus.cc:348] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.304912 30560 raft_consensus.cc:374] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:43.304932 30560 raft_consensus.cc:729] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3e2a385506448f9cfb6c1ef1b74184, State: Initialized, Role: FOLLOWER
I20260809 16:11:43.304939 30556 tablet_bootstrap.cc:492] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: No bootstrap required, opened a new log
I20260809 16:11:43.304977 30556 ts_tablet_manager.cc:1397] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:43.304992 30560 consensus_queue.cc:260] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.305115 30560 ts_tablet_manager.cc:1428] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:43.305081 30556 raft_consensus.cc:348] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.305145 30556 raft_consensus.cc:374] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:43.305161 30556 raft_consensus.cc:729] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0096f42e04394942012c0177703d7, State: Initialized, Role: FOLLOWER
I20260809 16:11:43.305202 30556 consensus_queue.cc:260] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.305300 30556 ts_tablet_manager.cc:1428] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:43.305580 30555 tablet_bootstrap.cc:492] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: No bootstrap required, opened a new log
I20260809 16:11:43.305631 30555 ts_tablet_manager.cc:1397] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:11:43.305776 30555 raft_consensus.cc:348] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.305836 30555 raft_consensus.cc:374] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:43.305855 30555 raft_consensus.cc:729] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Initialized, Role: FOLLOWER
I20260809 16:11:43.305893 30555 consensus_queue.cc:260] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.305990 30555 ts_tablet_manager.cc:1428] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:43.371142 30566 raft_consensus.cc:482] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:43.371241 30566 raft_consensus.cc:504] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.371411 30566 leader_election.cc:288] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:43.371632 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711d6e13bd7a49a78a37c8062a5a7d90" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7" is_pre_election: true
I20260809 16:11:43.371743 29967 raft_consensus.cc:2454] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 0.
I20260809 16:11:43.371793 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711d6e13bd7a49a78a37c8062a5a7d90" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" is_pre_election: true
I20260809 16:11:43.371922 30454 raft_consensus.cc:2454] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 0.
I20260809 16:11:43.371934 30102 leader_election.cc:302] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7; no voters:
I20260809 16:11:43.372032 30566 raft_consensus.cc:2781] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:43.372073 30566 raft_consensus.cc:482] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:43.372092 30566 raft_consensus.cc:3037] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.372761 30566 raft_consensus.cc:504] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.372900 30566 leader_election.cc:288] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 election: Requested vote from peers 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:43.373039 29967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711d6e13bd7a49a78a37c8062a5a7d90" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fa0096f42e04394942012c0177703d7"
I20260809 16:11:43.373122 29967 raft_consensus.cc:3037] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.373096 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711d6e13bd7a49a78a37c8062a5a7d90" candidate_uuid: "0946bd92b7984c2b97cfa60d287fec02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
I20260809 16:11:43.373188 30454 raft_consensus.cc:3037] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.373788 30454 raft_consensus.cc:2454] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 1.
I20260809 16:11:43.373968 30094 leader_election.cc:302] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 3d3e2a385506448f9cfb6c1ef1b74184; no voters:
I20260809 16:11:43.374058 30566 raft_consensus.cc:2781] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:43.374064 29967 raft_consensus.cc:2454] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0946bd92b7984c2b97cfa60d287fec02 in term 1.
I20260809 16:11:43.374114 30566 raft_consensus.cc:686] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 LEADER]: Becoming Leader. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Running, Role: LEADER
I20260809 16:11:43.374171 30566 consensus_queue.cc:237] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.374768 29686 catalog_manager.cc:5495] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "0946bd92b7984c2b97cfa60d287fec02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:43.381249 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.378845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47836 (local address 127.16.224.130:44411)
0809 16:11:43.378887 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.378889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.378900 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:43.378966 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.378966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.378975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.379020 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:43.379151 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.379604 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.380698 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.380878 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.380925 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.381076 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.381101 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.381131 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.381182 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.381198 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:11:43.381935 30454 raft_consensus.cc:1261] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Refusing update from remote peer 0946bd92b7984c2b97cfa60d287fec02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.381979 29967 raft_consensus.cc:1261] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Refusing update from remote peer 0946bd92b7984c2b97cfa60d287fec02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.382176 30566 consensus_queue.cc:1035] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.382334 30636 consensus_queue.cc:1035] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.384810 29686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:43.384909 29686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:43.385548 29686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Sending DeleteTablet for 3 replicas of tablet 711d6e13bd7a49a78a37c8062a5a7d90
I20260809 16:11:43.385771 29936 tablet_service.cc:1514] Processing DeleteTablet for tablet 711d6e13bd7a49a78a37c8062a5a7d90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:43618
I20260809 16:11:43.385809 29933 tablet_service.cc:1514] Processing DeleteTablet for tablet 711d6e13bd7a49a78a37c8062a5a7d90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:43618
I20260809 16:11:43.385871 30594 tablet_replica.cc:330] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: stopping tablet replica
I20260809 16:11:43.385931 30594 raft_consensus.cc:2229] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:43.385977 30594 raft_consensus.cc:2258] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.386034 30187 tablet_service.cc:1514] Processing DeleteTablet for tablet 711d6e13bd7a49a78a37c8062a5a7d90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:47812
I20260809 16:11:43.386116 30597 tablet_replica.cc:330] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: stopping tablet replica
I20260809 16:11:43.386173 30597 raft_consensus.cc:2229] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:43.386225 30597 raft_consensus.cc:2258] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.386224 29686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:11:43.386389 29686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:11:43.386548 30594 ts_tablet_manager.cc:1905] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.386548 30597 ts_tablet_manager.cc:1905] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.386608 30423 tablet_service.cc:1514] Processing DeleteTablet for tablet 711d6e13bd7a49a78a37c8062a5a7d90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:54040
I20260809 16:11:43.386687 30596 tablet_replica.cc:330] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: stopping tablet replica
I20260809 16:11:43.386729 30596 raft_consensus.cc:2229] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:43.386768 30596 raft_consensus.cc:2258] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.387256 30596 ts_tablet_manager.cc:1905] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.387987 30594 ts_tablet_manager.cc:1918] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:43.387987 30597 ts_tablet_manager.cc:1918] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:43.388036 30597 log.cc:1196] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/711d6e13bd7a49a78a37c8062a5a7d90
I20260809 16:11:43.388036 30594 log.cc:1196] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/711d6e13bd7a49a78a37c8062a5a7d90
I20260809 16:11:43.388239 30594 ts_tablet_manager.cc:1939] T 711d6e13bd7a49a78a37c8062a5a7d90 P 7fa0096f42e04394942012c0177703d7: Deleting consensus metadata
I20260809 16:11:43.388239 30597 ts_tablet_manager.cc:1939] T 711d6e13bd7a49a78a37c8062a5a7d90 P 0946bd92b7984c2b97cfa60d287fec02: Deleting consensus metadata
I20260809 16:11:43.388553 29665 catalog_manager.cc:4841] TS 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411): tablet 711d6e13bd7a49a78a37c8062a5a7d90 (table test-table [id=28b7bfdab0ea41bcac1e008f529a6705]) successfully deleted
I20260809 16:11:43.388649 30596 ts_tablet_manager.cc:1918] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:11:43.388700 30596 log.cc:1196] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/711d6e13bd7a49a78a37c8062a5a7d90
I20260809 16:11:43.388725 29668 catalog_manager.cc:4841] TS 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729): tablet 711d6e13bd7a49a78a37c8062a5a7d90 (table test-table [id=28b7bfdab0ea41bcac1e008f529a6705]) successfully deleted
W20260809 16:11:43.388842 29668 catalog_manager.cc:4808] TS 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729): delete failed for tablet 711d6e13bd7a49a78a37c8062a5a7d90 because the tablet was not found. No further retry: Not found: Tablet not found: 711d6e13bd7a49a78a37c8062a5a7d90
I20260809 16:11:43.388888 30596 ts_tablet_manager.cc:1939] T 711d6e13bd7a49a78a37c8062a5a7d90 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting consensus metadata
I20260809 16:11:43.389164 29664 catalog_manager.cc:4841] TS 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937): tablet 711d6e13bd7a49a78a37c8062a5a7d90 (table test-table [id=28b7bfdab0ea41bcac1e008f529a6705]) successfully deleted
I20260809 16:11:43.389957 29933 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 (DEFAULT_TABLE table=test-table [id=a01786990f9d47748fd383a06d5df19b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:43.389968 30187 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 (DEFAULT_TABLE table=test-table [id=a01786990f9d47748fd383a06d5df19b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:43.390064 30187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:43.390065 29933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:43.391358 30555 tablet_bootstrap.cc:492] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: Bootstrap starting.
I20260809 16:11:43.391407 30556 tablet_bootstrap.cc:492] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: Bootstrap starting.
I20260809 16:11:43.391700 30555 tablet_bootstrap.cc:654] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:43.391708 30556 tablet_bootstrap.cc:654] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:43.392127 30423 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 (DEFAULT_TABLE table=test-table [id=a01786990f9d47748fd383a06d5df19b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:11:43.392230 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:43.392428 30555 tablet_bootstrap.cc:492] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: No bootstrap required, opened a new log
I20260809 16:11:43.392462 30555 ts_tablet_manager.cc:1397] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:11:43.392566 30555 raft_consensus.cc:348] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.392611 30555 raft_consensus.cc:374] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:43.392623 30555 raft_consensus.cc:729] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0946bd92b7984c2b97cfa60d287fec02, State: Initialized, Role: FOLLOWER
I20260809 16:11:43.392658 30555 consensus_queue.cc:260] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.392742 30555 ts_tablet_manager.cc:1428] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:43.393364 30560 tablet_bootstrap.cc:492] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: Bootstrap starting.
I20260809 16:11:43.393639 30556 tablet_bootstrap.cc:492] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: No bootstrap required, opened a new log
I20260809 16:11:43.393682 30556 ts_tablet_manager.cc:1397] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260809 16:11:43.393746 30560 tablet_bootstrap.cc:654] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:43.393781 30556 raft_consensus.cc:348] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.393846 30556 raft_consensus.cc:374] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:43.393864 30556 raft_consensus.cc:729] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0096f42e04394942012c0177703d7, State: Initialized, Role: FOLLOWER
I20260809 16:11:43.393909 30556 consensus_queue.cc:260] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.394002 30556 ts_tablet_manager.cc:1428] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:43.394301 30560 tablet_bootstrap.cc:492] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: No bootstrap required, opened a new log
I20260809 16:11:43.394392 30560 ts_tablet_manager.cc:1397] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:43.394543 30560 raft_consensus.cc:348] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.394611 30560 raft_consensus.cc:374] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:43.394637 30560 raft_consensus.cc:729] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3e2a385506448f9cfb6c1ef1b74184, State: Initialized, Role: FOLLOWER
I20260809 16:11:43.394675 30560 consensus_queue.cc:260] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.394763 30560 ts_tablet_manager.cc:1428] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:43.626809 30576 raft_consensus.cc:482] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:11:43.626940 30576 raft_consensus.cc:504] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.627166 30576 leader_election.cc:288] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:43.627415 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9f02df94fb4787b61ba4d5f1bd10d9" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0946bd92b7984c2b97cfa60d287fec02" is_pre_election: true
I20260809 16:11:43.627406 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9f02df94fb4787b61ba4d5f1bd10d9" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" is_pre_election: true
I20260809 16:11:43.627562 30225 raft_consensus.cc:2454] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa0096f42e04394942012c0177703d7 in term 0.
I20260809 16:11:43.627610 30454 raft_consensus.cc:2454] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa0096f42e04394942012c0177703d7 in term 0.
I20260809 16:11:43.627744 29840 leader_election.cc:302] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7; no voters:
I20260809 16:11:43.627873 30576 raft_consensus.cc:2781] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:11:43.627928 30576 raft_consensus.cc:482] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:11:43.627947 30576 raft_consensus.cc:3037] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.628731 30576 raft_consensus.cc:504] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.628859 30576 leader_election.cc:288] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 election: Requested vote from peers 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411), 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937)
I20260809 16:11:43.629030 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9f02df94fb4787b61ba4d5f1bd10d9" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0946bd92b7984c2b97cfa60d287fec02"
I20260809 16:11:43.629025 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9f02df94fb4787b61ba4d5f1bd10d9" candidate_uuid: "7fa0096f42e04394942012c0177703d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3e2a385506448f9cfb6c1ef1b74184"
I20260809 16:11:43.629099 30225 raft_consensus.cc:3037] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.629102 30454 raft_consensus.cc:3037] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:43.629698 30225 raft_consensus.cc:2454] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fa0096f42e04394942012c0177703d7 in term 1.
I20260809 16:11:43.629756 30454 raft_consensus.cc:2454] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fa0096f42e04394942012c0177703d7 in term 1.
I20260809 16:11:43.629881 29840 leader_election.cc:302] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0946bd92b7984c2b97cfa60d287fec02, 7fa0096f42e04394942012c0177703d7; no voters:
I20260809 16:11:43.629966 30576 raft_consensus.cc:2781] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:43.630085 30576 raft_consensus.cc:686] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0096f42e04394942012c0177703d7, State: Running, Role: LEADER
I20260809 16:11:43.630141 30576 consensus_queue.cc:237] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } }
I20260809 16:11:43.630759 29686 catalog_manager.cc:5495] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0096f42e04394942012c0177703d7 (127.16.224.129). New cstate: current_term: 1 leader_uuid: "7fa0096f42e04394942012c0177703d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fa0096f42e04394942012c0177703d7" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 } health_report { overall_health: UNKNOWN } } }
I20260809 16:11:43.679807 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.677873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41286 (local address 127.16.224.129:44729)
0809 16:11:43.677986 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.677988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.678006 (+ 18us) server_negotiation.cc:407] Connection header received
0809 16:11:43.678029 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.678029 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.678040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.678095 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:43.678188 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.678681 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.679317 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.679498 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.679539 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.679627 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.679653 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.679681 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.679716 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.679730 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260809 16:11:43.685305 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.682398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:44411 (local address 127.16.224.129:47799)
0809 16:11:43.682819 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:43.682830 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:43.682888 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:43.683026 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:43.683026 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:43.683028 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:43.683071 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.683072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.683820 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.683820 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.684580 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.684583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.684921 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.684923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.684978 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.684995 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.685147 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.685149 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:11:43.685190 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:11:43.685235 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":85,"threads_started":1}
I20260809 16:11:43.685285 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.682719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:47799 (local address 127.16.224.130:44411)
0809 16:11:43.682766 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.682768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.682874 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:11:43.682937 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.682938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.682948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.683022 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:43.683144 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.683741 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.684661 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.684862 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.684915 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.685066 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.685099 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.685142 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.685193 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.685211 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:11:43.687139 30225 raft_consensus.cc:1261] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Refusing update from remote peer 7fa0096f42e04394942012c0177703d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.687143 30454 raft_consensus.cc:1261] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Refusing update from remote peer 7fa0096f42e04394942012c0177703d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:11:43.687337 30676 consensus_queue.cc:1035] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0946bd92b7984c2b97cfa60d287fec02" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.687464 30576 consensus_queue.cc:1035] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d3e2a385506448f9cfb6c1ef1b74184" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:11:43.693914 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.691873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:59689 (local address 127.16.224.129:44729)
0809 16:11:43.691921 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:43.691923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:43.691929 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:11:43.691948 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:43.691949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:43.691958 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:43.691999 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:11:43.692103 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.692542 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.693229 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.693486 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.693540 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.693694 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.693725 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.693768 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:43.693819 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:11:43.693839 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:11:43.693923 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.691730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44729 (local address 127.16.224.130:59689)
0809 16:11:43.691769 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:43.691780 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:43.691816 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:43.692016 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:43.692017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:43.692018 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:11:43.692056 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.692058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.692604 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.692605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.693154 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:43.693156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:43.693571 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:43.693573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:43.693628 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:43.693634 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:11:43.693774 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:11:43.693776 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:11:43.693817 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:11:43.693861 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 16:11:43.699608 29687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:43.699702 29687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:11:43.700306 29687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 522313375d2e40f6a07264f5e37f31bc: Sending DeleteTablet for 3 replicas of tablet 1e9f02df94fb4787b61ba4d5f1bd10d9
I20260809 16:11:43.700527 29933 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:43618
I20260809 16:11:43.700537 30187 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:47812
I20260809 16:11:43.700575 30423 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:11:43 UTC) from {username='oryx', principal='oryx/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:54040
I20260809 16:11:43.700640 30597 tablet_replica.cc:330] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: stopping tablet replica
I20260809 16:11:43.700656 30596 tablet_replica.cc:330] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: stopping tablet replica
I20260809 16:11:43.700690 30594 tablet_replica.cc:330] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: stopping tablet replica
I20260809 16:11:43.700704 30596 raft_consensus.cc:2229] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:43.700707 30597 raft_consensus.cc:2229] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:11:43.700737 30596 raft_consensus.cc:2258] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.700735 30594 raft_consensus.cc:2229] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:11:43.700740 30597 raft_consensus.cc:2258] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.700798 30594 raft_consensus.cc:2258] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:11:43.701078 30597 ts_tablet_manager.cc:1905] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.701297 30594 ts_tablet_manager.cc:1905] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.702450 30597 ts_tablet_manager.cc:1918] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:43.702454 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 29817
I20260809 16:11:43.702459 30594 ts_tablet_manager.cc:1918] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:43.702507 30597 log.cc:1196] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/1e9f02df94fb4787b61ba4d5f1bd10d9
I20260809 16:11:43.702491 30596 ts_tablet_manager.cc:1905] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:11:43.702512 30594 log.cc:1196] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/1e9f02df94fb4787b61ba4d5f1bd10d9
I20260809 16:11:43.702723 30597 ts_tablet_manager.cc:1939] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 0946bd92b7984c2b97cfa60d287fec02: Deleting consensus metadata
I20260809 16:11:43.702723 30594 ts_tablet_manager.cc:1939] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 7fa0096f42e04394942012c0177703d7: Deleting consensus metadata
I20260809 16:11:43.703755 30596 ts_tablet_manager.cc:1918] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:11:43.703806 30596 log.cc:1196] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/1e9f02df94fb4787b61ba4d5f1bd10d9
I20260809 16:11:43.703933 30596 ts_tablet_manager.cc:1939] T 1e9f02df94fb4787b61ba4d5f1bd10d9 P 3d3e2a385506448f9cfb6c1ef1b74184: Deleting consensus metadata
I20260809 16:11:43.705173 29665 catalog_manager.cc:4841] TS 0946bd92b7984c2b97cfa60d287fec02 (127.16.224.130:44411): tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 (table test-table [id=a01786990f9d47748fd383a06d5df19b]) successfully deleted
I20260809 16:11:43.707917 29664 catalog_manager.cc:4841] TS 3d3e2a385506448f9cfb6c1ef1b74184 (127.16.224.131:45937): tablet 1e9f02df94fb4787b61ba4d5f1bd10d9 (table test-table [id=a01786990f9d47748fd383a06d5df19b]) successfully deleted
W20260809 16:11:43.714879 29668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:44729 (error 108)
I20260809 16:11:43.715190 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 30067
I20260809 16:11:43.716778 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:43.715280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44729 (local address 127.0.0.1:41294)
0809 16:11:43.716669 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:43.716730 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1365,"thread_start_us":52,"threads_started":1}
W20260809 16:11:43.725292 29668 catalog_manager.cc:4584] TS 7fa0096f42e04394942012c0177703d7 (127.16.224.129:44729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e9f02df94fb4787b61ba4d5f1bd10d9: Network error: Client connection negotiation failed: client connection to 127.16.224.129:44729: connect: Connection refused (error 111)
I20260809 16:11:43.730391 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 30309
I20260809 16:11:43.741398 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 29640
2026-08-09T16:11:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2875 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:43 dist-test-slave-fgb3 krb5kdc[30704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:43 dist-test-slave-fgb3 krb5kdc[30704](info): set up 2 sockets
Aug 09 16:11:43 dist-test-slave-fgb3 krb5kdc[30704](info): commencing operation
krb5kdc: starting...
W20260809 16:11:44.891392 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:44 dist-test-slave-fgb3 krb5kdc[30704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:45.006531 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33927
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:33927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:45.088582 30717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:45.088789 30717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:45.088812 30717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:45.089932 30717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:45.089977 30717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:45.089990 30717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:45.090001 30717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:45.090013 30717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:45.091517 30717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:33927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:33927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:45.091749 30717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:45.091979 30717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:45.096662 30724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:45.096698 30725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:45.096748 30717 server_base.cc:1034] running on GCE node
W20260809 16:11:45.096698 30729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:45.097095 30717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:45.097379 30717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:45.098511 30717 hybrid_clock.cc:648] HybridClock initialized: now 1786291905098493 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T16:11:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1356 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:11:45 dist-test-slave-fgb3 krb5kdc[30738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:11:45 dist-test-slave-fgb3 krb5kdc[30738](info): set up 2 sockets
Aug 09 16:11:45 dist-test-slave-fgb3 krb5kdc[30738](info): commencing operation
krb5kdc: starting...
W20260809 16:11:46.237900 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:11:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:11:46.354619 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41617
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:41617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:46.436015 30754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:46.436215 30754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:46.436244 30754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:46.437369 30754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:46.437412 30754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:46.437431 30754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:46.437448 30754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:46.437463 30754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:46.438982 30754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:41617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:46.439229 30754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:46.439430 30754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:46.444051 30765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:46.444036 30767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:46.444041 30763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:46.444224 30754 server_base.cc:1034] running on GCE node
I20260809 16:11:46.444540 30754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:46.444818 30754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:46.445976 30754 hybrid_clock.cc:648] HybridClock initialized: now 1786291906445959 us; error 31 us; skew 500 ppm
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:46.448381 30754 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:46.448750 30754 webserver.cc:460] Webserver started at http://127.16.224.190:45843/ using document root <none> and password file <none>
I20260809 16:11:46.448927 30754 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:46.448990 30754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:46.449110 30754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:46.450104 30754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "668079fae5664be7aac6978e1d344222"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "d073b080bf5dd7449af75c6545c31e14"
server_key_iv: "aff93e5604cab97d5e6e09ede72baeaf"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.450459 30754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "668079fae5664be7aac6978e1d344222"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "d073b080bf5dd7449af75c6545c31e14"
server_key_iv: "aff93e5604cab97d5e6e09ede72baeaf"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.451700 30754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:46.452379 30780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.452531 30754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:11:46.452620 30754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "668079fae5664be7aac6978e1d344222"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "d073b080bf5dd7449af75c6545c31e14"
server_key_iv: "aff93e5604cab97d5e6e09ede72baeaf"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.452679 30754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:46.468427 30754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:46.470054 30754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:46.470183 30754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:46.473868 30754 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:41617
I20260809 16:11:46.473910 30862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:41617 every 8 connection(s)
I20260809 16:11:46.474208 30754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:46.475069 30865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:11:46.477245 30865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Bootstrap starting.
I20260809 16:11:46.477782 30865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Neither blocks nor log segments found. Creating new log.
I20260809 16:11:46.478029 30865 log.cc:824] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:46.478744 30865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: No bootstrap required, opened a new log
I20260809 16:11:46.479468 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 30754
I20260809 16:11:46.479564 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:11:46.479749 17282 external_mini_cluster.cc:1428] Setting key fa599aaa9577fd6eb0dd764f6fe9343e
I20260809 16:11:46.480512 30865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:46.480654 30865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:11:46.480687 30865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668079fae5664be7aac6978e1d344222, State: Initialized, Role: FOLLOWER
I20260809 16:11:46.480810 30865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:46.480903 30865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:46.480940 30865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:46.480989 30865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:11:46.481916 30865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:46.482004 30865 leader_election.cc:302] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 668079fae5664be7aac6978e1d344222; no voters:
I20260809 16:11:46.482159 30865 leader_election.cc:288] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:11:46.482209 30871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:11:46.482366 30871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 LEADER]: Becoming Leader. State: Replica: 668079fae5664be7aac6978e1d344222, State: Running, Role: LEADER
I20260809 16:11:46.482424 30865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:46.482452 30871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:46.483486 30874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "668079fae5664be7aac6978e1d344222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } } }
I20260809 16:11:46.483566 30874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:46.483773 30875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 668079fae5664be7aac6978e1d344222. Latest consensus state: current_term: 1 leader_uuid: "668079fae5664be7aac6978e1d344222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } } }
I20260809 16:11:46.483850 30875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: This master's current role is: LEADER
W20260809 16:11:46.484306 30900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:11:46.484892 30900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:11:46.484966 30896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:46.485137 30896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:46.486498 30896 catalog_manager.cc:1347] Generated new cluster ID: 04ca1a545fdd46f3bea12a90d2a3eed7
I20260809 16:11:46.486548 30896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:46.490502 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:46.480217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.16.224.190:41617)
0809 16:11:46.480388 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:46.480395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:46.480420 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:11:46.480513 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:46.480519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:46.480557 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:46.480665 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:46.481433 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.482571 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.483633 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.484151 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.484255 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:46.486753 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:46.486756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:46.486757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:46.486767 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:46.488510 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.488896 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.488898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.488899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.489014 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.489200 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.489201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.489201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.489475 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:46.490085 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:46.490151 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:11:46.490286 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260809 16:11:46.515998 30896 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:11:46.516525 30896 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:46.521464 30896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Generated new TSK 0
I20260809 16:11:46.521636 30896 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:11:46.546834 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41617
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:46.627767 30912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:46.627978 30912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:46.628003 30912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:46.629114 30912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:46.629145 30912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:46.629169 30912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:46.630586 30912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:46.630798 30912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:46.630986 30912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:46.635949 30921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:46.635944 30918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:46.635974 30912 server_base.cc:1034] running on GCE node
W20260809 16:11:46.635944 30919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:46.636379 30912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:46.636603 30912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:46.637732 30912 hybrid_clock.cc:648] HybridClock initialized: now 1786291906637715 us; error 33 us; skew 500 ppm
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:46.640311 30912 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:11:46.640740 30912 webserver.cc:460] Webserver started at http://127.16.224.129:34623/ using document root <none> and password file <none>
I20260809 16:11:46.640985 30912 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:46.641029 30912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:46.641163 30912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:46.642212 30912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "49ffe8885d38445faf0624d529e2fd68"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "fc31a329110faca5eb2dc4e2b6da5076"
server_key_iv: "3f3f9c63bbd3aa0aaf68c261ef301f5c"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.642614 30912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "49ffe8885d38445faf0624d529e2fd68"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "fc31a329110faca5eb2dc4e2b6da5076"
server_key_iv: "3f3f9c63bbd3aa0aaf68c261ef301f5c"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.643966 30912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:46.644788 30951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.644950 30912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:46.645040 30912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "49ffe8885d38445faf0624d529e2fd68"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "fc31a329110faca5eb2dc4e2b6da5076"
server_key_iv: "3f3f9c63bbd3aa0aaf68c261ef301f5c"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.645109 30912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:46.666270 30912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:46.668560 30912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:46.668711 30912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:46.668947 30912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:46.669266 30912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:46.669301 30912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.669330 30912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:46.669353 30912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.674829 30912 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:42437
I20260809 16:11:46.674881 31147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:42437 every 8 connection(s)
I20260809 16:11:46.675199 30912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:46.681792 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 30912
I20260809 16:11:46.681893 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:11:46.682097 17282 external_mini_cluster.cc:1428] Setting key d61b89033b25868fc107eec89cf07a5c
I20260809 16:11:46.684665 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:46.675910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:48369 (local address 127.16.224.190:41617)
0809 16:11:46.675978 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:46.675980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:46.676163 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:11:46.676674 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:46.676675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:46.676689 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:46.676731 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:46.677654 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.678291 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.679445 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.679670 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.679727 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:46.681744 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:46.681747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:46.681747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:46.681756 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:46.683269 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.683727 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.683728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.683728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.683778 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.684045 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.684046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.684046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.684148 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:46.684197 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:46.684483 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 16:11:46.684570 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:11:46.684872 31152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:46.675862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:48369)
0809 16:11:46.676042 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:46.676086 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:46.676576 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:46.676801 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:46.676805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:46.677387 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:46.677584 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:46.677590 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.678394 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.678395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:46.679352 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:46.679355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.679744 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.679745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:46.679878 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:46.680307 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:46.680316 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:46.681632 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:46.683355 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:46.683365 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:46.683367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:46.683655 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:46.683837 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:46.683838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:46.683838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:46.683992 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:46.684200 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:46.684205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:46.684363 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 16:11:46.684471 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:11:46.684644 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260809 16:11:46.685528 31148 heartbeater.cc:347] Connected to a master server at 127.16.224.190:41617
I20260809 16:11:46.685623 31148 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:46.685798 31148 heartbeater.cc:511] Master 127.16.224.190:41617 requested a full tablet report, sending...
I20260809 16:11:46.686349 30812 ts_manager.cc:194] Registered new tserver with Master: 49ffe8885d38445faf0624d529e2fd68 (127.16.224.129:42437)
I20260809 16:11:46.686931 30812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:48369
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:11:46.706918 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41617
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:46.788201 31159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:46.788384 31159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:46.788405 31159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:46.789466 31159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:46.789500 31159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:46.789525 31159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:46.790915 31159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:46.791097 31159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:46.791268 31159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:46.795912 31170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:46.795903 31174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:46.795974 31159 server_base.cc:1034] running on GCE node
W20260809 16:11:46.795918 31169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:46.796247 31159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:46.796465 31159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:46.797609 31159 hybrid_clock.cc:648] HybridClock initialized: now 1786291906797596 us; error 31 us; skew 500 ppm
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:46.800102 31159 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:11:46.800565 31159 webserver.cc:460] Webserver started at http://127.16.224.130:43349/ using document root <none> and password file <none>
I20260809 16:11:46.800804 31159 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:46.800858 31159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:46.800997 31159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:46.802088 31159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "18019b810aec4bf9b2f3a7b9db7788ce"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "59e278843015ff4795ef0817bfdbb6c1"
server_key_iv: "58d5ee08c545221f27dc927bd275ea8a"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.802481 31159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "18019b810aec4bf9b2f3a7b9db7788ce"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "59e278843015ff4795ef0817bfdbb6c1"
server_key_iv: "58d5ee08c545221f27dc927bd275ea8a"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.803779 31159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:46.804453 31188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.804602 31159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:46.804692 31159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "18019b810aec4bf9b2f3a7b9db7788ce"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "59e278843015ff4795ef0817bfdbb6c1"
server_key_iv: "58d5ee08c545221f27dc927bd275ea8a"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.804764 31159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:46.817111 31159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:46.819036 31159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:46.819175 31159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:46.819379 31159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:46.819671 31159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:46.819708 31159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.819738 31159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:46.819777 31159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.825985 31159 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:41411
I20260809 16:11:46.826026 31386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:41411 every 8 connection(s)
I20260809 16:11:46.826378 31159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:11:46.832052 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 31159
I20260809 16:11:46.832150 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:46.832401 17282 external_mini_cluster.cc:1428] Setting key 73c852ae1a3fd56dbfc5223d95f19ceb
I20260809 16:11:46.835436 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:46.827116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:36295 (local address 127.16.224.190:41617)
0809 16:11:46.827194 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:46.827196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:46.827371 (+ 175us) server_negotiation.cc:407] Connection header received
0809 16:11:46.827881 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:46.827883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:46.827897 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:46.827950 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:46.828921 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.829429 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.830443 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.830640 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.830700 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:46.832870 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:46.832872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:46.832873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:46.832880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:46.834145 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.834570 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.834571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.834571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.834625 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.834845 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.834846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.834846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.834954 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:46.835010 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:46.835263 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 16:11:46.835347 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:11:46.835611 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:46.827077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:36295)
0809 16:11:46.827246 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:46.827283 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:46.827773 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:46.828023 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:46.828027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:46.828609 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:46.828815 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:46.828821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.829515 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.829516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:46.830351 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:46.830354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.830711 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.830712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:46.830834 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:46.831303 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:46.831319 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:46.832761 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:46.834222 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:46.834229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:46.834231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:46.834505 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:46.834679 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:46.834679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:46.834680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:46.834785 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:46.835009 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:46.835017 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:46.835150 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 16:11:46.835237 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:11:46.835400 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260809 16:11:46.836050 31388 heartbeater.cc:347] Connected to a master server at 127.16.224.190:41617
I20260809 16:11:46.836153 31388 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:46.836334 31388 heartbeater.cc:511] Master 127.16.224.190:41617 requested a full tablet report, sending...
I20260809 16:11:46.836730 30812 ts_manager.cc:194] Registered new tserver with Master: 18019b810aec4bf9b2f3a7b9db7788ce (127.16.224.130:41411)
I20260809 16:11:46.837219 30812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:36295
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:11:46.855247 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41617
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:46.935596 31399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:46.935791 31399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:46.935819 31399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:46.936937 31399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:46.936977 31399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:46.937011 31399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:46.938445 31399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:46.938686 31399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:46.938889 31399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:46.943635 31409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:46.943732 31399 server_base.cc:1034] running on GCE node
W20260809 16:11:46.943616 31404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:46.943615 31407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:46.944067 31399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:46.944303 31399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:46.945454 31399 hybrid_clock.cc:648] HybridClock initialized: now 1786291906945437 us; error 29 us; skew 500 ppm
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:46.947863 31399 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:11:46.948302 31399 webserver.cc:460] Webserver started at http://127.16.224.131:46125/ using document root <none> and password file <none>
I20260809 16:11:46.948536 31399 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:46.948585 31399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:46.948711 31399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:11:46.949726 31399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "f13eda08312645a78d636b726848ef0f"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "4c7961c24604ea2a957a17ce258c753d"
server_key_iv: "d484b2431d1c50d598b0cd8d3b3a2623"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.950049 31399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "f13eda08312645a78d636b726848ef0f"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "4c7961c24604ea2a957a17ce258c753d"
server_key_iv: "d484b2431d1c50d598b0cd8d3b3a2623"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.951334 31399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:11:46.952020 31417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.952170 31399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:11:46.952260 31399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "f13eda08312645a78d636b726848ef0f"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "4c7961c24604ea2a957a17ce258c753d"
server_key_iv: "d484b2431d1c50d598b0cd8d3b3a2623"
server_key_version: "encryptionkey@0"
I20260809 16:11:46.952342 31399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:46.964229 31399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:46.966010 31399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:46.966146 31399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:46.966364 31399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:46.966646 31399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:46.966679 31399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.966708 31399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:46.966725 31399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:46.972153 31399 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:39521
I20260809 16:11:46.972214 31605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:39521 every 8 connection(s)
I20260809 16:11:46.972570 31399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:11:46 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291906, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:11:46.979662 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 31399
I20260809 16:11:46.979768 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:11:46.979981 17282 external_mini_cluster.cc:1428] Setting key 66534be86c2ec000bf503de40fa65f17
I20260809 16:11:46.981454 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:46.973180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:53103 (local address 127.16.224.190:41617)
0809 16:11:46.973265 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:46.973267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:46.973477 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:11:46.973988 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:46.973989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:46.974001 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:46.974050 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:46.975012 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.975504 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.976467 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.976653 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.976707 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:46.978665 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:46.978666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:46.978666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:46.978674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:46.980045 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.980527 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.980528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.980529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.980597 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:46.980885 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:46.980886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:46.980887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:46.981030 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:46.981088 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:46.981325 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 16:11:46.981392 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 16:11:46.981596 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:46.973139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:53103)
0809 16:11:46.973333 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:46.973403 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:46.973895 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:46.974131 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:46.974135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:46.974720 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:46.974914 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:46.974920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.975587 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.975588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:46.976381 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:46.976384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:46.976720 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:46.976721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:46.976843 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:46.977252 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:46.977260 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:46.978578 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:11:46.980137 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:46.980145 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:46.980147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:46.980436 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:46.980668 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:11:46.980669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:11:46.980669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:11:46.980812 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:11:46.981087 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:11:46.981092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:11:46.981207 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 16:11:46.981306 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 16:11:46.981423 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260809 16:11:46.982013 31606 heartbeater.cc:347] Connected to a master server at 127.16.224.190:41617
I20260809 16:11:46.982098 31606 heartbeater.cc:464] Registering TS with master...
I20260809 16:11:46.982266 31606 heartbeater.cc:511] Master 127.16.224.190:41617 requested a full tablet report, sending...
I20260809 16:11:46.982659 30812 ts_manager.cc:194] Registered new tserver with Master: f13eda08312645a78d636b726848ef0f (127.16.224.131:39521)
I20260809 16:11:46.983011 30812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:53103
I20260809 16:11:46.991644 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab.
Entry for principal oryx/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab.
WARNING: no policy specified for oryx/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab.
Entry for principal oryx/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab.
WARNING: no policy specified for oryx/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab.
Entry for principal oryx/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab.
I20260809 16:11:47.060608 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 30912
I20260809 16:11:47.072527 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 31159
I20260809 16:11:47.086719 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 31399
I20260809 16:11:47.093990 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 30754
I20260809 16:11:47.103181 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41617
--webserver_interface=127.16.224.190
--webserver_port=45843
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:41617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:11:47.183945 31617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:47.184139 31617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:47.184160 31617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:47.185272 31617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:11:47.185313 31617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:47.185325 31617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:47.185336 31617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:11:47.185350 31617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:11:47.186810 31617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:41617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=45843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:47.187058 31617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:47.187285 31617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:47.191802 31628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:47.191793 31632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:47.191847 31617 server_base.cc:1034] running on GCE node
W20260809 16:11:47.191794 31629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:47.192265 31617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:47.192536 31617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:47.193675 31617 hybrid_clock.cc:648] HybridClock initialized: now 1786291907193661 us; error 27 us; skew 500 ppm
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291907, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:47.200672 31617 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:11:47.201097 31617 webserver.cc:460] Webserver started at http://127.16.224.190:45843/ using document root <none> and password file <none>
I20260809 16:11:47.201269 31617 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:47.201320 31617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:47.202606 31617 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:47.203218 31644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.203363 31617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.203446 31617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "668079fae5664be7aac6978e1d344222"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "d073b080bf5dd7449af75c6545c31e14"
server_key_iv: "aff93e5604cab97d5e6e09ede72baeaf"
server_key_version: "encryptionkey@0"
I20260809 16:11:47.203780 31617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:47.218443 31617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:47.220304 31617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:47.220432 31617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:47.224342 31617 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:41617
I20260809 16:11:47.224386 31724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:41617 every 8 connection(s)
I20260809 16:11:47.224731 31617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:11:47.226181 31725 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:11:47.226843 31725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Bootstrap starting.
I20260809 16:11:47.227973 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 31617
I20260809 16:11:47.228250 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:42437
--local_ip_for_outbound_sockets=127.16.224.129
--tserver_master_addrs=127.16.224.190:41617
--webserver_port=34623
--webserver_interface=127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:11:47.235193 31725 log.cc:824] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Log is configured to *not* fsync() on all Append() calls
I20260809 16:11:47.236773 31725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:11:47.237025 31725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Bootstrap complete.
I20260809 16:11:47.239197 31725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:47.239418 31725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668079fae5664be7aac6978e1d344222, State: Initialized, Role: FOLLOWER
I20260809 16:11:47.239573 31725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:47.239650 31725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:11:47.239696 31725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:11:47.239743 31725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:11:47.240688 31725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:47.240785 31725 leader_election.cc:302] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 668079fae5664be7aac6978e1d344222; no voters:
I20260809 16:11:47.240949 31725 leader_election.cc:288] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:11:47.240993 31730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:11:47.241158 31730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [term 2 LEADER]: Becoming Leader. State: Replica: 668079fae5664be7aac6978e1d344222, State: Running, Role: LEADER
I20260809 16:11:47.241218 31725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:11:47.241694 31730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } }
I20260809 16:11:47.242074 31734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 668079fae5664be7aac6978e1d344222. Latest consensus state: current_term: 2 leader_uuid: "668079fae5664be7aac6978e1d344222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } } }
I20260809 16:11:47.242147 31734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:47.242422 31731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "668079fae5664be7aac6978e1d344222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668079fae5664be7aac6978e1d344222" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41617 } } }
I20260809 16:11:47.242533 31731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222 [sys.catalog]: This master's current role is: LEADER
I20260809 16:11:47.242981 31761 catalog_manager.cc:1259] Loaded cluster ID: 04ca1a545fdd46f3bea12a90d2a3eed7
I20260809 16:11:47.243031 31761 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: loading cluster ID for follower catalog manager: success
I20260809 16:11:47.244204 31761 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: acquiring CA information for follower catalog manager: success
I20260809 16:11:47.244534 31761 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:11:47.244665 31755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:11:47.244858 31755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:11:47.244930 31755 catalog_manager.cc:1259] Loaded cluster ID: 04ca1a545fdd46f3bea12a90d2a3eed7
I20260809 16:11:47.244958 31755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:11:47.245167 31755 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:11:47.245265 31755 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 668079fae5664be7aac6978e1d344222: Loaded TSK: 0
I20260809 16:11:47.245410 31755 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:11:47.311844 31728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:47.312134 31728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:47.312168 31728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:47.313921 31728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:47.313975 31728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:47.314014 31728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:11:47.316092 31728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.129.keytab
--principal=oryx/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:42437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=34623
--tserver_master_addrs=127.16.224.190:41617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:47.316345 31728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:47.316627 31728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:47.322777 31773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:47.326815 31768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:47.328739 31728 server_base.cc:1034] running on GCE node
W20260809 16:11:47.328965 31767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:47.329241 31728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:47.329546 31728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:47.330705 31728 hybrid_clock.cc:648] HybridClock initialized: now 1786291907330675 us; error 45 us; skew 500 ppm
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291907, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:47.333169 31728 init.cc:377] Logged in from keytab as oryx/127.16.224.129@KRBTEST.COM (short username oryx)
I20260809 16:11:47.333603 31728 webserver.cc:460] Webserver started at http://127.16.224.129:34623/ using document root <none> and password file <none>
I20260809 16:11:47.333827 31728 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:47.333899 31728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:47.335265 31728 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:47.335831 31788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.336007 31728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.336107 31728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "49ffe8885d38445faf0624d529e2fd68"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "fc31a329110faca5eb2dc4e2b6da5076"
server_key_iv: "3f3f9c63bbd3aa0aaf68c261ef301f5c"
server_key_version: "encryptionkey@0"
I20260809 16:11:47.336588 31728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:47.349814 31728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:47.352226 31728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:47.352391 31728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:47.352656 31728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:47.353030 31728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:47.353081 31728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.353121 31728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:47.353143 31728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.360400 31728 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:42437
I20260809 16:11:47.360432 31979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:42437 every 8 connection(s)
I20260809 16:11:47.360853 31728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:11:47.367122 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 31728
I20260809 16:11:47.367421 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:41411
--local_ip_for_outbound_sockets=127.16.224.130
--tserver_master_addrs=127.16.224.190:41617
--webserver_port=43349
--webserver_interface=127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.370247 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.361589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:57471)
0809 16:11:47.361770 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.361814 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.362455 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.362851 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.362855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.363541 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.363800 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.363807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.364868 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.364869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.366092 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.366095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.366704 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.366705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.366853 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.367390 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.367400 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.370047 (+ 2647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
W20260809 16:11:47.370298 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.361702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:57471 (local address 127.16.224.190:41617)
0809 16:11:47.361883 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.361890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.361916 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:11:47.362629 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.362635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.362677 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.362784 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.363908 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.364775 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.366174 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.366629 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.366705 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.370172 (+ 3467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:57471: BlockingRecv error: recv got EOF from 127.16.224.129:57471 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
W20260809 16:11:47.370798 31980 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:41617 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.224.190:41617: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.376116 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.371019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:40809)
0809 16:11:47.371117 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.371126 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.371167 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.371307 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.371308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.371447 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.371501 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.371503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.372352 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.372353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.373121 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.373124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.373573 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.373574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.373639 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.374065 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.374071 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.376051 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 16:11:47.376435 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.371077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:40809 (local address 127.16.224.190:41617)
0809 16:11:47.371123 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.371126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.371158 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:11:47.371213 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.371215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.371226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.371288 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.371573 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.372259 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.373204 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.373502 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.373563 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.376141 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:40809: BlockingRecv error: recv got EOF from 127.16.224.129:40809 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.380996 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.376348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:58593)
0809 16:11:47.376612 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.376622 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.376662 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.376812 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.376812 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.376941 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.376974 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.376977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.377709 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.377710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.378497 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.378500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.378895 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.378896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.378952 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.379346 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.379351 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.380934 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":69,"threads_started":1}
W20260809 16:11:47.381067 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.376500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:58593 (local address 127.16.224.190:41617)
0809 16:11:47.376656 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.376658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.376668 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:47.376730 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.376731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.376741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.376796 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.377046 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.377629 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.378572 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.378813 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.378881 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.381014 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:58593: BlockingRecv error: recv got EOF from 127.16.224.129:58593 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
W20260809 16:11:47.381130 31980 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:11:47.451774 31988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:47.452010 31988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:47.452035 31988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:47.453181 31988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:47.453255 31988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:47.453291 31988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:11:47.454780 31988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.130.keytab
--principal=oryx/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:41411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=43349
--tserver_master_addrs=127.16.224.190:41617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:47.454982 31988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:47.455174 31988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:47.460007 31996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:47.460005 32000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:47.460002 31997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:47.460248 31988 server_base.cc:1034] running on GCE node
I20260809 16:11:47.460445 31988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:47.460683 31988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:47.461856 31988 hybrid_clock.cc:648] HybridClock initialized: now 1786291907461839 us; error 28 us; skew 500 ppm
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291907, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:47.464284 31988 init.cc:377] Logged in from keytab as oryx/127.16.224.130@KRBTEST.COM (short username oryx)
I20260809 16:11:47.464680 31988 webserver.cc:460] Webserver started at http://127.16.224.130:43349/ using document root <none> and password file <none>
I20260809 16:11:47.464884 31988 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:47.464949 31988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:47.466284 31988 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:11:47.466861 32013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.467010 31988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.467103 31988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "18019b810aec4bf9b2f3a7b9db7788ce"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "59e278843015ff4795ef0817bfdbb6c1"
server_key_iv: "58d5ee08c545221f27dc927bd275ea8a"
server_key_version: "encryptionkey@0"
I20260809 16:11:47.467442 31988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:47.477052 31988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:47.479100 31988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:47.479230 31988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:47.479437 31988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:47.479733 31988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:47.479766 31988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.479794 31988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:47.479818 31988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.485532 31988 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:41411
I20260809 16:11:47.485611 32205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:41411 every 8 connection(s)
I20260809 16:11:47.485903 31988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.492465 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.486550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:52989)
0809 16:11:47.486685 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.486719 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.487178 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.487399 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.487402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.487939 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.488123 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.488128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.488796 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.488797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.489583 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.489585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.489866 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.489866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.489957 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.490418 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.490427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.492324 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
W20260809 16:11:47.492470 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.486560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:52989 (local address 127.16.224.190:41617)
0809 16:11:47.486602 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.486604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.486810 (+ 206us) server_negotiation.cc:407] Connection header received
0809 16:11:47.487270 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.487271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.487287 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.487335 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.488221 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.488730 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.489641 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.489812 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.489857 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.492422 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:52989: BlockingRecv error: recv got EOF from 127.16.224.130:52989 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:11:47.492690 32206 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:41617 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.224.190:41617: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
I20260809 16:11:47.493898 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 31988
I20260809 16:11:47.494177 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:39521
--local_ip_for_outbound_sockets=127.16.224.131
--tserver_master_addrs=127.16.224.190:41617
--webserver_port=46125
--webserver_interface=127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.496853 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.492883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:51647)
0809 16:11:47.492927 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.492938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.492978 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.493080 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.493081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.493213 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.493259 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.493260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.493903 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.493904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.494572 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.494574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.494959 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.494960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.495011 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.495361 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.495365 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.496805 (+ 1440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:11:47.496943 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.492927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:51647 (local address 127.16.224.190:41617)
0809 16:11:47.492982 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.492983 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.493006 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:11:47.493026 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.493027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.493037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.493078 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.493322 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.493820 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.494645 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.494888 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.494955 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.496889 (+ 1934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:51647: BlockingRecv error: recv got EOF from 127.16.224.130:51647 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.501160 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.497093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:37737)
0809 16:11:47.497134 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.497146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.497186 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.497303 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.497303 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.497411 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.497436 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.497437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.498176 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.498176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.498720 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.498722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.499167 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.499168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.499214 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.499597 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.499601 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.501115 (+ 1514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:11:47.501271 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.497153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:37737 (local address 127.16.224.190:41617)
0809 16:11:47.497203 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.497205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.497225 (+ 20us) server_negotiation.cc:407] Connection header received
0809 16:11:47.497244 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.497245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.497254 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.497295 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.497490 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.498111 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.498800 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.499069 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.499139 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.501213 (+ 2074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:37737: BlockingRecv error: recv got EOF from 127.16.224.130:37737 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 16:11:47.501425 32206 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:11:47.576498 32211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:11:47.576684 32211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:11:47.576707 32211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:11:47.577760 32211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:11:47.577791 32211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:11:47.577816 32211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:11:47.579224 32211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/krb5kdc/oryx_127.16.224.131.keytab
--principal=oryx/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:39521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=46125
--tserver_master_addrs=127.16.224.190:41617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:11:47.579396 32211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:11:47.579562 32211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:11:47.584091 32223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:47.584091 32220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:11:47.584161 32221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:11:47.584400 32211 server_base.cc:1034] running on GCE node
I20260809 16:11:47.584599 32211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:11:47.584800 32211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:11:47.585963 32211 hybrid_clock.cc:648] HybridClock initialized: now 1786291907585951 us; error 26 us; skew 500 ppm
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291907, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:11:47.588495 32211 init.cc:377] Logged in from keytab as oryx/127.16.224.131@KRBTEST.COM (short username oryx)
I20260809 16:11:47.588953 32211 webserver.cc:460] Webserver started at http://127.16.224.131:46125/ using document root <none> and password file <none>
I20260809 16:11:47.589157 32211 fs_manager.cc:359] Metadata directory not provided
I20260809 16:11:47.589215 32211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:11:47.590534 32211 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:47.591223 32231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.591400 32211 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:11:47.591498 32211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "f13eda08312645a78d636b726848ef0f"
format_stamp: "Formatted at 2026-08-09 16:11:46 on dist-test-slave-fgb3"
server_key: "4c7961c24604ea2a957a17ce258c753d"
server_key_iv: "d484b2431d1c50d598b0cd8d3b3a2623"
server_key_version: "encryptionkey@0"
I20260809 16:11:47.591812 32211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:11:47.602694 32211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:11:47.604595 32211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:11:47.604696 32211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:11:47.604869 32211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:11:47.605098 32211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:11:47.605127 32211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.605146 32211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:11:47.605161 32211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:11:47.610639 32211 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:39521
I20260809 16:11:47.610700 32411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:39521 every 8 connection(s)
I20260809 16:11:47.611002 32211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.617722 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.611673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:34193)
0809 16:11:47.611846 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.611878 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.612344 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.612599 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.612603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.613144 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.613326 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.613331 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.613988 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.613989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.614844 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.614847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.615191 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.615192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.615305 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.615671 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.615680 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.617591 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
W20260809 16:11:47.617723 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.611697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:34193 (local address 127.16.224.190:41617)
0809 16:11:47.611746 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.611748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.611959 (+ 211us) server_negotiation.cc:407] Connection header received
0809 16:11:47.612450 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.612451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.612464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.612515 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.613428 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.613906 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.614930 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.615122 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.615178 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.617677 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:34193: BlockingRecv error: recv got EOF from 127.16.224.131:34193 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:11:47.617980 32415 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:41617 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.224.190:41617: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
I20260809 16:11:47.620276 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 32211
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.622144 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.618191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:36085)
0809 16:11:47.618233 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.618242 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.618278 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.618477 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.618477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.618579 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.618623 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.618624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.619259 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.619260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.619759 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.619760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.620028 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.620028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.620071 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.620445 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.620450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.622084 (+ 1634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 16:11:47.622223 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.618350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:36085 (local address 127.16.224.190:41617)
0809 16:11:47.618385 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.618387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.618393 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:11:47.618411 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.618412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.618420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.618462 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.618716 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.619176 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.619817 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.619980 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.620021 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.622174 (+ 2153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:36085: BlockingRecv error: recv got EOF from 127.16.224.131:36085 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:47 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:47.626221 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.622422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:57061)
0809 16:11:47.622468 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:47.622477 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:47.622518 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:47.622634 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:47.622635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:47.622749 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:47.622773 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.622774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.623331 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.623332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.623810 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:47.623811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.624229 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.624230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:47.624266 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.624611 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:47.624615 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:47.626164 (+ 1549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:11:47.626287 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:47.622481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:57061 (local address 127.16.224.190:41617)
0809 16:11:47.622516 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.622518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.622525 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:47.622572 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.622573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.622581 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.622626 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.622833 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.623258 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.623886 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.624158 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.624232 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.626243 (+ 2011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:57061: BlockingRecv error: recv got EOF from 127.16.224.131:57061 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:11:47.626344 32415 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:11:47.627477 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:47.620790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.16.224.190:41617)
0809 16:11:47.620859 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:47.620860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:47.620868 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:47.620927 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:47.620928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:47.620938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:47.621000 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:47.621582 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.622042 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.623095 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:47.623300 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:47.623359 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:47.624590 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:11:47.624596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:11:47.624596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:11:47.624609 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:11:47.626226 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:47.626714 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:47.626717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:47.626717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:47.626811 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:11:47.626963 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:11:47.626964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:11:47.626964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:11:47.627160 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:11:47.627252 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:11:47.627335 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 16:11:47.627421 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 16:11:48 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:48 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:48.385838 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:48.381509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:45897)
0809 16:11:48.381633 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:48.381644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:48.381696 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:48.381820 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:48.381821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:48.381980 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:48.382031 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:48.382033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.382708 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.382709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:48.383264 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:48.383266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.383591 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.383591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:48.383627 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:48.384113 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:48.384118 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:48.385775 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260809 16:11:48.385982 32425 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:48.381591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:45897 (local address 127.16.224.190:41617)
0809 16:11:48.381709 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:48.381711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:48.381719 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:48.381740 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:48.381741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:48.381751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:48.381815 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:48.382090 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.382652 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.383339 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.383537 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.383588 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:48.385903 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:45897: BlockingRecv error: recv got EOF from 127.16.224.129:45897 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
Aug 09 16:11:48 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:48 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:48.506354 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:48.501813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:46609)
0809 16:11:48.501985 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:48.501999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:48.502070 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:48.502273 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:48.502274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:48.502522 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:48.502581 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:48.502584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.503278 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.503279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:48.503817 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:48.503818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.504144 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.504144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:48.504180 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:48.504595 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:48.504600 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:48.506253 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":61,"threads_started":1}
W20260809 16:11:48.506443 32425 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:48.501904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:46609 (local address 127.16.224.190:41617)
0809 16:11:48.501971 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:48.501974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:48.502052 (+ 78us) server_negotiation.cc:407] Connection header received
0809 16:11:48.502126 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:48.502127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:48.502142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:48.502214 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:48.502668 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.503192 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.503893 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.504086 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.504145 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:48.506383 (+ 2238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:46609: BlockingRecv error: recv got EOF from 127.16.224.130:46609 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:11:48 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:48 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:48.631394 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:48.626704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:59733)
0809 16:11:48.626836 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:48.626846 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:48.626906 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:48.627044 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:48.627045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:48.627209 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:48.627259 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:48.627260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.627958 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.627959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:48.628521 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:48.628523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.628858 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.628858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:48.628894 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:48.629370 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:48.629375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:48.631324 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
W20260809 16:11:48.631484 32425 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:48.626759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:59733 (local address 127.16.224.190:41617)
0809 16:11:48.626859 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:48.626861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:48.626893 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:11:48.626962 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:48.626963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:48.626980 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:48.627033 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:48.627332 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.627856 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.628595 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:48.628800 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:48.628855 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:48.631419 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:59733: BlockingRecv error: recv got EOF from 127.16.224.131:59733 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 16:11:49 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:49 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:49.390663 32430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:49.386272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:51761)
0809 16:11:49.386438 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:49.386451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:49.386516 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:49.386649 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:49.386650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:49.386803 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:49.386853 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:49.386855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.387499 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.387500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:49.388032 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:49.388034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.388336 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.388337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:49.388381 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:49.388833 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:49.388838 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:49.390602 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
W20260809 16:11:49.390785 32432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:49.386380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:51761 (local address 127.16.224.190:41617)
0809 16:11:49.386530 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:49.386532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:49.386541 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:49.386565 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:49.386565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:49.386577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:49.386641 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:49.386908 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.387421 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.388094 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.388282 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.388330 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:49.390712 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:51761: BlockingRecv error: recv got EOF from 127.16.224.129:51761 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
Aug 09 16:11:49 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:49 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:49.511380 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:49.506799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:44671)
0809 16:11:49.506955 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:49.506966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:49.507034 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:49.507157 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:49.507158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:49.507389 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:49.507439 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:49.507442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.508094 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.508096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:49.508646 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:49.508649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.508987 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.508988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:49.509044 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:49.509490 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:49.509496 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:49.511320 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
W20260809 16:11:49.511482 32432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:49.506886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:44671 (local address 127.16.224.190:41617)
0809 16:11:49.506956 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:49.506958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:49.507012 (+ 54us) server_negotiation.cc:407] Connection header received
0809 16:11:49.507095 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:49.507096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:49.507109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:49.507151 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:49.507513 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.508019 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.508725 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.508924 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.508978 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:49.511412 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:44671: BlockingRecv error: recv got EOF from 127.16.224.130:44671 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 16:11:49 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:49 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:49.636461 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:49.631902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:48191)
0809 16:11:49.632065 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:49.632079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:49.632140 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:49.632277 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:49.632279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:49.632465 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:49.632512 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:49.632513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.633151 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.633152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:49.633686 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:49.633687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.634008 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.634010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:49.634063 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:49.634576 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:49.634581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:49.636380 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
W20260809 16:11:49.636554 32432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:49.632098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:48191 (local address 127.16.224.190:41617)
0809 16:11:49.632163 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:49.632165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:49.632173 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:49.632198 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:49.632199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:49.632214 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:49.632263 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:49.632584 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.633076 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.633760 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:49.633946 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:49.634001 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:49.636496 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:48191: BlockingRecv error: recv got EOF from 127.16.224.131:48191 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 16:11:50 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:50 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:50.395650 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:50.391189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:53733)
0809 16:11:50.391333 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:50.391346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:50.391404 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:50.391560 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:50.391561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:50.391726 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:50.391785 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:50.391787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.392420 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.392420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:50.392945 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:50.392947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.393279 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.393280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:50.393344 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:50.393827 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:50.393832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:50.395577 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
W20260809 16:11:50.395740 32440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:50.391237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:53733 (local address 127.16.224.190:41617)
0809 16:11:50.391381 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:50.391382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:50.391393 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:50.391484 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:50.391485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:50.391498 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:50.391545 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:50.391836 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.392358 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.393007 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.393185 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.393227 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:50.395666 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:53733: BlockingRecv error: recv got EOF from 127.16.224.129:53733 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
Aug 09 16:11:50 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:50 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:50.516141 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:50.511835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:33381)
0809 16:11:50.511994 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:50.512007 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:50.512076 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:50.512207 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:50.512208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:50.512380 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:50.512430 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:50.512432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.513095 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.513096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:50.513636 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:50.513638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.513927 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.513928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:50.513963 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:50.514441 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:50.514445 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:50.516081 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
W20260809 16:11:50.516239 32440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:50.511942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:33381 (local address 127.16.224.190:41617)
0809 16:11:50.512011 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:50.512013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:50.512058 (+ 45us) server_negotiation.cc:407] Connection header received
0809 16:11:50.512122 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:50.512123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:50.512134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:50.512188 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:50.512487 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.513019 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.513693 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.513878 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.513923 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:50.516173 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:33381: BlockingRecv error: recv got EOF from 127.16.224.130:33381 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:11:50 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:50 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:50.641672 32443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:50.636963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:36693)
0809 16:11:50.637099 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:50.637110 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:50.637169 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:50.637305 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:50.637306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:50.637470 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:50.637521 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:50.637523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.638132 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.638133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:50.638725 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:50.638727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.639135 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.639137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:50.639206 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:50.639664 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:50.639669 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:50.641589 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260809 16:11:50.641773 32440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:50.637044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:36693 (local address 127.16.224.190:41617)
0809 16:11:50.637102 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:50.637104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:50.637164 (+ 60us) server_negotiation.cc:407] Connection header received
0809 16:11:50.637211 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:50.637212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:50.637224 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:50.637285 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:50.637569 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.638073 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.638819 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:50.639037 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:50.639107 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:50.641713 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:36693: BlockingRecv error: recv got EOF from 127.16.224.131:36693 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 16:11:51 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:51 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:51.400528 32444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:51.396091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:42869)
0809 16:11:51.396213 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:51.396223 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:51.396280 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:51.396418 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:51.396418 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:51.396593 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:51.396646 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:51.396649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.397289 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.397292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:51.397942 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:51.397944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.398262 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.398263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:51.398340 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:51.398755 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:51.398759 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:51.400447 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
W20260809 16:11:51.400610 32445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:51.396134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:42869 (local address 127.16.224.190:41617)
0809 16:11:51.396259 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:51.396261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:51.396271 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:51.396331 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:51.396332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:51.396343 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:51.396411 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:51.396699 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.397203 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.398016 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.398207 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.398253 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:51.400546 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:42869: BlockingRecv error: recv got EOF from 127.16.224.129:42869 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
Aug 09 16:11:51 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:51 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:51.520905 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:51.516589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:51979)
0809 16:11:51.516723 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:51.516736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:51.516807 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:51.516953 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:51.516954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:51.517134 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:51.517182 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:51.517184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.517837 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.517839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:51.518410 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:51.518412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.518735 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.518736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:51.518790 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:51.519183 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:51.519187 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:51.520827 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
W20260809 16:11:51.520973 32445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:51.516659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:51979 (local address 127.16.224.190:41617)
0809 16:11:51.516727 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:51.516729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:51.516796 (+ 67us) server_negotiation.cc:407] Connection header received
0809 16:11:51.516878 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:51.516879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:51.516893 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:51.516942 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:51.517254 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.517758 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.518486 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.518677 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.518731 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:51.520919 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:51979: BlockingRecv error: recv got EOF from 127.16.224.130:51979 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 16:11:51 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:51 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:51.646234 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:51.642112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:56247)
0809 16:11:51.642247 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:51.642257 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:51.642339 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:51.642481 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:51.642482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:51.642646 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:51.642696 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:51.642698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.643301 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.643302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:51.643838 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:51.643840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.644108 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.644108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:51.644147 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:51.644541 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:51.644545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:51.646170 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
W20260809 16:11:51.646370 32445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:51.642180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:56247 (local address 127.16.224.190:41617)
0809 16:11:51.642238 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:51.642240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:51.642306 (+ 66us) server_negotiation.cc:407] Connection header received
0809 16:11:51.642397 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:51.642398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:51.642413 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:51.642474 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:51.642747 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.643239 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.643894 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:51.644060 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:51.644102 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:51.646276 (+ 2174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:56247: BlockingRecv error: recv got EOF from 127.16.224.131:56247 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 16:11:52 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:52 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:52.408195 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:52.400963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:57903)
0809 16:11:52.401088 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:52.401099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:52.401166 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:52.401330 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:52.401331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:52.401541 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:52.401603 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:52.401607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.402265 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.402266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:52.402842 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:52.402844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.403179 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.403179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:52.403216 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:52.406406 (+ 3190us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:52.406412 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:52.408134 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
W20260809 16:11:52.408303 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:52.401024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:57903 (local address 127.16.224.190:41617)
0809 16:11:52.401135 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:52.401137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:52.401153 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:11:52.401219 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:52.401220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:52.401234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:52.401297 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:52.401681 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.402195 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.402917 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.403123 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.403176 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:52.408221 (+ 5045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:57903: BlockingRecv error: recv got EOF from 127.16.224.129:57903 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":55,"threads_started":1}
Aug 09 16:11:52 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:52 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:52.526592 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:52.521380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:40073)
0809 16:11:52.522479 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:52.522492 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:52.522553 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:52.522695 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:52.522696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:52.522860 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:52.522919 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:52.522921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.523546 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.523546 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:52.524089 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:52.524091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.524379 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.524379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:52.524419 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:52.524868 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:52.524872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:52.526532 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1080,"thread_start_us":1047,"threads_started":1}
W20260809 16:11:52.526690 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:52.521526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:40073 (local address 127.16.224.190:41617)
0809 16:11:52.521578 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:52.521580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:52.522591 (+ 1011us) server_negotiation.cc:407] Connection header received
0809 16:11:52.522624 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:52.522625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:52.522640 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:52.522682 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:52.522970 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.523481 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.524147 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.524331 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.524373 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:52.526630 (+ 2257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:40073: BlockingRecv error: recv got EOF from 127.16.224.130:40073 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 16:11:52 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:52 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:52.651455 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:52.646749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:37811)
0809 16:11:52.646874 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:52.646888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:52.646997 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:52.647142 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:52.647143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:52.647365 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:52.647418 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:52.647420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.648143 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.648145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:52.648714 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:52.648717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.649074 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.649074 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:52.649113 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:52.649560 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:52.649566 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:52.651376 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
W20260809 16:11:52.651542 32453 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:52.646860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:37811 (local address 127.16.224.190:41617)
0809 16:11:52.646938 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:52.646940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:52.646982 (+ 42us) server_negotiation.cc:407] Connection header received
0809 16:11:52.647060 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:52.647061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:52.647079 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:52.647136 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:52.647499 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.648069 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.648804 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:52.649007 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:52.649066 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:52.651480 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:37811: BlockingRecv error: recv got EOF from 127.16.224.131:37811 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 16:11:53 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:53 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:53.413201 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:53.408672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:56835)
0809 16:11:53.408812 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:53.408823 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:53.408871 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:53.409028 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:53.409029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:53.409188 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:53.409258 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:53.409260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.409915 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.409915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:53.410465 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:53.410467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.410895 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.410896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:53.410970 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:53.411397 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:53.411411 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:53.413121 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
W20260809 16:11:53.413280 32457 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:53.408764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:56835 (local address 127.16.224.190:41617)
0809 16:11:53.408905 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:53.408907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:53.408915 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:11:53.408938 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:53.408939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:53.408951 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:53.409021 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:53.409322 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.409850 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.410540 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.410814 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.410883 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:53.413219 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:56835: BlockingRecv error: recv got EOF from 127.16.224.129:56835 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
Aug 09 16:11:53 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:53 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:53.531625 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:53.527077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:34923)
0809 16:11:53.527201 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:53.527212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:53.527269 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:53.527394 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:53.527395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:53.527559 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:53.527609 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:53.527611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.528356 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.528357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:53.528940 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:53.528943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.529284 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.529285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:53.529346 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:53.529846 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:53.529851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:53.531565 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
W20260809 16:11:53.531710 32457 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:53.527187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:34923 (local address 127.16.224.190:41617)
0809 16:11:53.527247 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:53.527248 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:53.527262 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:11:53.527323 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:53.527324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:53.527335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:53.527381 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:53.527705 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.528243 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.529023 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.529211 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.529285 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:53.531649 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:34923: BlockingRecv error: recv got EOF from 127.16.224.130:34923 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 16:11:53 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:53 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:53.656349 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:53.651916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:40643)
0809 16:11:53.652072 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:53.652083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:53.652138 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:53.652260 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:53.652261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:53.652429 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:53.652482 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:53.652484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.653115 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.653116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:53.653665 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:53.653667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.654010 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.654011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:53.654049 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:53.654568 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:53.654573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:53.656287 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
W20260809 16:11:53.656520 32457 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:53.651983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:40643 (local address 127.16.224.190:41617)
0809 16:11:53.652030 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:53.652032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:53.652123 (+ 91us) server_negotiation.cc:407] Connection header received
0809 16:11:53.652194 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:53.652195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:53.652207 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:53.652245 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:53.652540 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.653041 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.653749 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:53.653955 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:53.654008 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:53.656451 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:40643: BlockingRecv error: recv got EOF from 127.16.224.131:40643 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 16:11:54 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:54 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:54.417985 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:54.413661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:41861)
0809 16:11:54.413874 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:54.413887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:54.413960 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:54.414122 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:54.414122 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:54.414292 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:54.414375 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:54.414378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.414986 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.414987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:54.415510 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:54.415511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.415797 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.415797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:54.415833 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:54.416256 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:54.416260 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:54.417924 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260809 16:11:54.418103 32462 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:54.413723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:41861 (local address 127.16.224.190:41617)
0809 16:11:54.413832 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:54.413834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:54.413954 (+ 120us) server_negotiation.cc:407] Connection header received
0809 16:11:54.414036 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:54.414037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:54.414051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:54.414102 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:54.414432 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.414928 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.415563 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.415749 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.415792 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:54.418034 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:41861: BlockingRecv error: recv got EOF from 127.16.224.129:41861 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
Aug 09 16:11:54 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:54 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:54.537124 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:54.532071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:58239)
0809 16:11:54.532208 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:54.532221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:54.532305 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:54.532427 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:54.532428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:54.532589 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:54.532646 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:54.532648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.533600 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.533601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:54.534192 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:54.534194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.534590 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.534590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:54.534636 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:54.535041 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:54.535046 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:54.537044 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
W20260809 16:11:54.537220 32462 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:54.532169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:58239 (local address 127.16.224.190:41617)
0809 16:11:54.532225 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:54.532227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:54.532270 (+ 43us) server_negotiation.cc:407] Connection header received
0809 16:11:54.532364 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:54.532364 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:54.532376 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:54.532413 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:54.532703 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.533483 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.534279 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.534527 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.534582 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:54.537157 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:58239: BlockingRecv error: recv got EOF from 127.16.224.130:58239 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 16:11:54 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:54 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:54.661412 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:54.656878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:51639)
0809 16:11:54.657038 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:54.657050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:54.657111 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:54.657280 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:54.657281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:54.657456 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:54.657508 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:54.657509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.658184 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.658185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:54.658791 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:54.658793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.659086 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.659087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:54.659124 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:54.659540 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:54.659544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:54.661353 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
W20260809 16:11:54.661546 32462 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:54.656999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:51639 (local address 127.16.224.190:41617)
0809 16:11:54.657064 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:54.657066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:54.657098 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:11:54.657174 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:54.657175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:54.657195 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:54.657251 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:54.657581 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.658094 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.658857 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:54.659036 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:54.659081 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:54.661478 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:51639: BlockingRecv error: recv got EOF from 127.16.224.131:51639 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:11:55 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:55 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:55.423338 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:55.418698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:59811)
0809 16:11:55.418858 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:55.418870 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:55.418924 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:55.419062 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:55.419063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:55.419249 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:55.419307 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:55.419309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.419996 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.419998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:55.420571 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:55.420573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.420929 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.420929 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:55.420967 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:55.421428 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:55.421434 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:55.423273 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
W20260809 16:11:55.423506 32469 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:55.418826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:59811 (local address 127.16.224.190:41617)
0809 16:11:55.418945 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:55.418947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:55.418954 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:11:55.418977 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:55.418977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:55.418988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:55.419047 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:55.419383 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.419900 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.420659 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.420870 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.420925 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:55.423426 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:59811: BlockingRecv error: recv got EOF from 127.16.224.129:59811 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
Aug 09 16:11:55 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:55 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:55.542771 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:55.537914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:56173)
0809 16:11:55.538067 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:55.538081 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:55.538154 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:55.538298 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:55.538299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:55.538521 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:55.538569 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:55.538572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.539378 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.539378 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:55.539910 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:55.539912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.540262 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.540262 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:55.540318 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:55.540791 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:55.540796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:55.542679 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
W20260809 16:11:55.542841 32469 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:55.537994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:56173 (local address 127.16.224.190:41617)
0809 16:11:55.538083 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:55.538085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:55.538145 (+ 60us) server_negotiation.cc:407] Connection header received
0809 16:11:55.538221 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:55.538222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:55.538235 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:55.538282 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:55.538646 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.539296 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.539988 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.540189 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.540245 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:55.542787 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:56173: BlockingRecv error: recv got EOF from 127.16.224.130:56173 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 16:11:55 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:55 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:55.666257 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:55.661921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:44569)
0809 16:11:55.662079 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:55.662090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:55.662154 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:55.662276 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:55.662277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:55.662461 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:55.662511 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:55.662513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.663128 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.663128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:55.663645 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:55.663646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.663953 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.663954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:55.663993 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:55.664400 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:55.664405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:55.666187 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
W20260809 16:11:55.666364 32469 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:55.662017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:44569 (local address 127.16.224.190:41617)
0809 16:11:55.662083 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:55.662085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:55.662147 (+ 62us) server_negotiation.cc:407] Connection header received
0809 16:11:55.662200 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:55.662201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:55.662215 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:55.662261 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:55.662568 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.663073 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.663722 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:55.663902 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:55.663946 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:55.666279 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:44569: BlockingRecv error: recv got EOF from 127.16.224.131:44569 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 16:11:56 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:56 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:56.428351 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:56.423842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:48239)
0809 16:11:56.423977 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:56.423990 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:56.424042 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:56.424213 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:56.424214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:56.424384 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:56.424435 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:56.424437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.425149 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.425150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:56.425696 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:56.425698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.426019 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.426020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:56.426056 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:56.426549 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:56.426554 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:56.428284 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260809 16:11:56.428476 32475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:56.423877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:48239 (local address 127.16.224.190:41617)
0809 16:11:56.424018 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:56.424020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:56.424031 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:11:56.424091 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:56.424092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:56.424104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:56.424165 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:56.424502 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.425053 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.425767 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.425963 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.426016 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:56.428393 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:48239: BlockingRecv error: recv got EOF from 127.16.224.129:48239 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
Aug 09 16:11:56 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:56 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:56.547827 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:56.543273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:54905)
0809 16:11:56.543429 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:56.543439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:56.543501 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:56.543622 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:56.543623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:56.543784 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:56.543843 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:56.543845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.544499 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.544500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:56.545050 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:56.545051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.545453 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.545454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:56.545505 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:56.545960 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:56.545965 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:56.547763 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260809 16:11:56.547969 32475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:56.543347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:54905 (local address 127.16.224.190:41617)
0809 16:11:56.543416 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:56.543418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:56.543493 (+ 75us) server_negotiation.cc:407] Connection header received
0809 16:11:56.543549 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:56.543550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:56.543560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:56.543610 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:56.543890 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.544394 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.545144 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.545362 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.545436 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:56.547897 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:54905: BlockingRecv error: recv got EOF from 127.16.224.130:54905 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 16:11:56 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:56 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:56.671299 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:56.666735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:50421)
0809 16:11:56.666866 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:56.666877 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:56.666941 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:56.667058 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:56.667059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:56.667212 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:56.667264 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:56.667265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.667863 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.667863 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:56.668385 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:56.668386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.668670 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.668671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:56.668724 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:56.669282 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:56.669288 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:56.671225 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
W20260809 16:11:56.671379 32475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:56.666857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:50421 (local address 127.16.224.190:41617)
0809 16:11:56.666916 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:56.666918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:56.666928 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:56.666992 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:56.666992 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:56.667005 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:56.667052 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:56.667310 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.667808 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.668441 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:56.668617 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:56.668660 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:56.671316 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:50421: BlockingRecv error: recv got EOF from 127.16.224.131:50421 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:11:57 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:57 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:57.438059 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:57.428856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:52127)
0809 16:11:57.429012 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:57.429026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:57.429102 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:57.429255 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:57.429256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:57.429437 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:57.429494 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:57.429496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.430141 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.430143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:57.430914 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:57.430918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.431241 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.431241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:57.431282 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:57.436179 (+ 4897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:57.436185 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:57.437965 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
W20260809 16:11:57.438154 32481 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:57.428932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:52127 (local address 127.16.224.190:41617)
0809 16:11:57.429085 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:57.429087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:57.429096 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:57.429161 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:57.429161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:57.429173 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:57.429230 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:57.429555 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.430061 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.430978 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.431181 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.431236 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:57.438092 (+ 6856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:52127: BlockingRecv error: recv got EOF from 127.16.224.129:52127 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
Aug 09 16:11:57 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:57 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:57.552718 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:57.548296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:38645)
0809 16:11:57.548463 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:57.548473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:57.548542 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:57.548691 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:57.548692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:57.548873 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:57.548931 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:57.548934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.549606 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.549607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:57.550127 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:57.550128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.550499 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.550500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:57.550556 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:57.550977 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:57.550981 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:57.552660 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
W20260809 16:11:57.552826 32481 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:57.548356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:38645 (local address 127.16.224.190:41617)
0809 16:11:57.548407 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:57.548409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:57.548527 (+ 118us) server_negotiation.cc:407] Connection header received
0809 16:11:57.548610 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:57.548611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:57.548624 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:57.548672 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:57.549002 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.549528 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.550197 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.550429 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.550480 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:57.552755 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:38645: BlockingRecv error: recv got EOF from 127.16.224.130:38645 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 16:11:57 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:57 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:57.676254 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:57.671760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:43071)
0809 16:11:57.671979 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:57.671991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:57.672075 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:57.672240 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:57.672241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:57.672475 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:57.672540 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:57.672542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.673283 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.673284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:57.673815 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:57.673816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.674097 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.674098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:57.674133 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:57.674571 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:57.674575 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:57.676187 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":70,"threads_started":1}
W20260809 16:11:57.676410 32481 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:57.671891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:43071 (local address 127.16.224.190:41617)
0809 16:11:57.671969 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:57.671972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:57.672052 (+ 80us) server_negotiation.cc:407] Connection header received
0809 16:11:57.672139 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:57.672140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:57.672157 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:57.672219 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:57.672608 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.673213 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.673875 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:57.674053 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:57.674093 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:57.676289 (+ 2196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:43071: BlockingRecv error: recv got EOF from 127.16.224.131:43071 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:11:58 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:58 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:58.443131 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:58.438587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:53165)
0809 16:11:58.438716 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:58.438727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:58.438776 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:58.438913 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:58.438914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:58.439088 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:58.439134 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:58.439136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.439759 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.439760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:58.440285 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:58.440287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.440665 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.440666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:58.440708 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:58.441180 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:58.441185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:58.443058 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
W20260809 16:11:58.443217 32488 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:58.438647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:53165 (local address 127.16.224.190:41617)
0809 16:11:58.438779 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:58.438781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:58.438791 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:11:58.438823 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:58.438824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:58.438835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:58.438904 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:58.439195 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.439703 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.440370 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.440607 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.440665 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:58.443152 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:53165: BlockingRecv error: recv got EOF from 127.16.224.129:53165 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 09 16:11:58 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:58 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:58.557724 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:58.553219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:58573)
0809 16:11:58.553375 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:58.553385 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:58.553444 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:58.553572 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:58.553573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:58.553740 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:58.553787 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:58.553789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.554448 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.554449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:58.555040 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:58.555043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.555508 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.555509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:58.555556 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:58.555947 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:58.555951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:58.557653 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
W20260809 16:11:58.557896 32488 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:58.553287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:58573 (local address 127.16.224.190:41617)
0809 16:11:58.553344 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:58.553346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:58.553430 (+ 84us) server_negotiation.cc:407] Connection header received
0809 16:11:58.553500 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:58.553500 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:58.553511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:58.553560 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:58.553842 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.554393 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.555133 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.555434 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.555487 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:58.557844 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:58573: BlockingRecv error: recv got EOF from 127.16.224.130:58573 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 16:11:58 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:58 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:58.680935 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:58.676750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:33013)
0809 16:11:58.676918 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:58.676930 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:58.676988 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:58.677114 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:58.677114 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:58.677289 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:58.677338 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:58.677339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.677933 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.677933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:58.678519 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:58.678520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.678818 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.678819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:58.678853 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:58.679292 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:58.679296 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:58.680860 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":62,"threads_started":1}
W20260809 16:11:58.680996 32488 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:58.676882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:33013 (local address 127.16.224.190:41617)
0809 16:11:58.676944 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:58.676946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:58.676974 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:11:58.677035 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:58.677036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:58.677050 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:58.677101 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:58.677386 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.677881 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.678583 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:58.678771 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:58.678812 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:58.680946 (+ 2134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:33013: BlockingRecv error: recv got EOF from 127.16.224.131:33013 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 16:11:59 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:59 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:59.447942 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:59.443571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:54575)
0809 16:11:59.443701 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:59.443712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:59.443771 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:59.443890 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:59.443891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:59.444060 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:59.444111 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:59.444113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.444721 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.444721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:59.445246 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:59.445247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.445623 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.445624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:59.445685 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:59.446136 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:59.446141 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:59.447882 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
W20260809 16:11:59.448050 32492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:59.443667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:54575 (local address 127.16.224.190:41617)
0809 16:11:59.443776 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:59.443778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:59.443787 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:11:59.443813 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:59.443813 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:59.443824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:59.443878 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:59.444163 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.444668 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.445303 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.445552 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.445609 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:59.447973 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:54575: BlockingRecv error: recv got EOF from 127.16.224.129:54575 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":48,"threads_started":1}
Aug 09 16:11:59 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:59 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:59.562587 32495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:59.558178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:55461)
0809 16:11:59.558364 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:59.558376 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:59.558441 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:59.558559 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:59.558560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:59.558732 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:59.558784 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:59.558786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.559398 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.559399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:59.559936 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:59.559937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.560264 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.560267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:59.560322 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:59.560756 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:59.560761 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:59.562529 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":71,"threads_started":1}
W20260809 16:11:59.562685 32492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:59.558261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:55461 (local address 127.16.224.190:41617)
0809 16:11:59.558358 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:59.558360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:59.558428 (+ 68us) server_negotiation.cc:407] Connection header received
0809 16:11:59.558488 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:59.558489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:59.558500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:59.558546 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:59.558836 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.559336 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.559991 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.560169 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.560212 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:59.562621 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:55461: BlockingRecv error: recv got EOF from 127.16.224.130:55461 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 16:11:59 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:11:59 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:11:59.685747 32496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:11:59.681342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:47265)
0809 16:11:59.681475 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:11:59.681487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:11:59.681563 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:11:59.681702 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:11:59.681703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:11:59.681884 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:11:59.681931 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:59.681933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.682647 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.682648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:59.683199 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:11:59.683201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.683529 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.683529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:11:59.683583 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:59.683996 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:11:59.684001 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:11:59.685670 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
W20260809 16:11:59.685834 32492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:11:59.681427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:47265 (local address 127.16.224.190:41617)
0809 16:11:59.681499 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:11:59.681501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:11:59.681541 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:11:59.681620 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:11:59.681621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:11:59.681636 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:11:59.681679 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:11:59.682005 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.682570 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.683279 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:11:59.683470 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:11:59.683523 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:11:59.685772 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:47265: BlockingRecv error: recv got EOF from 127.16.224.131:47265 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 16:12:00 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:00 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:00.452736 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:00.448362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:34133)
0809 16:12:00.448493 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:00.448504 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:00.448553 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:00.448747 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:00.448748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:00.448917 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:00.448967 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:00.448969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.449619 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.449619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:00.450138 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:00.450139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.450531 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.450532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:00.450571 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:00.450997 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:00.451001 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:00.452677 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
W20260809 16:12:00.452884 32499 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:00.448461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:34133 (local address 127.16.224.190:41617)
0809 16:12:00.448615 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:00.448616 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:00.448626 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:12:00.448649 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:00.448650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:00.448664 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:00.448731 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:00.449021 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.449555 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.450194 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.450462 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.450531 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:00.452809 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:34133: BlockingRecv error: recv got EOF from 127.16.224.129:34133 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
Aug 09 16:12:00 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:00 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:00.567517 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:00.563085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:50495)
0809 16:12:00.563243 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:00.563257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:00.563324 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:00.563470 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:00.563471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:00.563650 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:00.563696 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:00.563698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.564352 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.564353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:00.564890 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:00.564892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.565229 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.565230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:00.565286 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:00.565734 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:00.565738 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:00.567439 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
W20260809 16:12:00.567600 32499 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:00.563184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:50495 (local address 127.16.224.190:41617)
0809 16:12:00.563247 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:00.563249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:00.563329 (+ 80us) server_negotiation.cc:407] Connection header received
0809 16:12:00.563391 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:00.563392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:00.563406 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:00.563452 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:00.563770 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.564267 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.564969 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.565168 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.565223 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:00.567546 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:50495: BlockingRecv error: recv got EOF from 127.16.224.130:50495 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:12:00 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:00 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:00.691016 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:00.686399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:36001)
0809 16:12:00.686588 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:00.686600 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:00.686672 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:00.686911 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:00.686912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:00.687085 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:00.687136 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:00.687137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.687767 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.687768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:00.688323 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:00.688325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.688689 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.688689 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:00.688725 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:00.689155 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:00.689159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:00.690958 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
W20260809 16:12:00.691124 32499 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:00.686482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:36001 (local address 127.16.224.190:41617)
0809 16:12:00.686550 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:00.686552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:00.686658 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:12:00.686722 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:00.686722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:00.686736 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:00.686795 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:00.687201 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.687696 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.688429 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:00.688628 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:00.688688 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:00.691052 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:36001: BlockingRecv error: recv got EOF from 127.16.224.131:36001 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 16:12:01 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:01 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:01.457823 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:01.453280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:57413)
0809 16:12:01.453466 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:01.453482 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:01.453561 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:01.453698 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:01.453699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:01.453872 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:01.453927 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:01.453928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.454568 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.454569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:01.455134 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:01.455136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.455517 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.455518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:01.455585 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:01.456030 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:01.456035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:01.457749 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
W20260809 16:12:01.457919 32508 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:01.453407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:57413 (local address 127.16.224.190:41617)
0809 16:12:01.453557 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:01.453559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:01.453568 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:12:01.453612 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:01.453612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:01.453624 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:01.453683 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:01.453980 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.454516 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.455213 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.455441 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.455511 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:01.457844 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:57413: BlockingRecv error: recv got EOF from 127.16.224.129:57413 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
Aug 09 16:12:01 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:01 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:01.572412 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:01.568024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:50477)
0809 16:12:01.568162 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:01.568173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:01.568232 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:01.568354 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:01.568355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:01.568533 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:01.568581 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:01.568583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.569234 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.569236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:01.569822 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:01.569824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.570185 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.570186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:01.570262 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:01.570699 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:01.570704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:01.572340 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
W20260809 16:12:01.572535 32508 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:01.568091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:50477 (local address 127.16.224.190:41617)
0809 16:12:01.568148 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:01.568150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:01.568223 (+ 73us) server_negotiation.cc:407] Connection header received
0809 16:12:01.568279 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:01.568280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:01.568291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:01.568343 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:01.568630 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.569131 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.569895 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.570092 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.570151 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:01.572468 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:50477: BlockingRecv error: recv got EOF from 127.16.224.130:50477 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 16:12:01 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:01 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.131@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:01.696148 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:01.691535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.131:45465)
0809 16:12:01.691719 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:01.691731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:01.691798 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:01.691966 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:01.691967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:01.692162 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:01.692212 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:01.692214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.692896 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.692898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:01.693460 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:01.693462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.693744 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.693745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:01.693781 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:01.694268 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:01.694273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:01.696066 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":93,"threads_started":1}
W20260809 16:12:01.696218 32508 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:01.691611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:45465 (local address 127.16.224.190:41617)
0809 16:12:01.691662 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:01.691665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:01.691799 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:12:01.691863 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:01.691863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:01.691890 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:01.691945 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:01.692310 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.692818 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.693521 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:01.693693 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:01.693750 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:01.696157 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.131:45465: BlockingRecv error: recv got EOF from 127.16.224.131:45465 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 16:12:02 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:02 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.129@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:02.465438 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:02.458305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.129:37063)
0809 16:12:02.458462 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:02.458473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:02.458522 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:02.458640 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:02.458641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:02.458805 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:02.458885 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:02.458887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.459494 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.459494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:02.460036 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:02.460037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.460384 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.460384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:02.460433 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:02.463509 (+ 3076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:02.463514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:02.465370 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
W20260809 16:12:02.465538 32515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:02.458400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:37063 (local address 127.16.224.190:41617)
0809 16:12:02.458521 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:02.458523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:02.458532 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:12:02.458557 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:02.458558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:02.458570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:02.458627 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:02.458942 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.459441 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.460110 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.460328 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.460382 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:02.465463 (+ 5081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.129:37063: BlockingRecv error: recv got EOF from 127.16.224.129:37063 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
Aug 09 16:12:02 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:12:02 dist-test-slave-fgb3 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.224.130@KRBTEST.COM for oryx/127.16.224.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:12:02.577348 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:02.572963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41617 (local address 127.16.224.130:48633)
0809 16:12:02.573136 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:02.573147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:02.573232 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:02.573360 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:02.573361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:02.573525 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:02.573574 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:02.573576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.574202 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.574202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:02.574751 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:02.574753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.575108 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.575109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:02.575143 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:02.575594 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:02.575598 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:02.577288 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.224.190:41617: Server oryx/127.16.224.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":68,"threads_started":1}
W20260809 16:12:02.577468 32515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:02.573023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:48633 (local address 127.16.224.190:41617)
0809 16:12:02.573102 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:02.573104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:02.573228 (+ 124us) server_negotiation.cc:407] Connection header received
0809 16:12:02.573282 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:02.573283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:02.573297 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:02.573345 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:02.573644 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.574145 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.574834 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.575042 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.575111 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:02.577399 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.224.130:48633: BlockingRecv error: recv got EOF from 127.16.224.130:48633 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:12:02.624214 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 31728
I20260809 16:12:02.656023 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 31988
I20260809 16:12:02.674109 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 32211
I20260809 16:12:02.681571 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 31617
2026-08-09T16:12:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17583 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T16:12:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:12:02Z Disabled control of system clock
I20260809 16:12:02.715634 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41941
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:41941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:12:02.797012 32526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:02.797207 32526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:02.797226 32526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:02.798426 32526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:12:02.798471 32526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:02.798482 32526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:02.798493 32526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:12:02.798504 32526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:12:02.799903 32526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:41941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:02.800159 32526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:02.800447 32526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:02.805198 32534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:02.805207 32526 server_base.cc:1034] running on GCE node
W20260809 16:12:02.805188 32537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:02.805172 32535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:02.805678 32526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:02.805960 32526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:02.807099 32526 hybrid_clock.cc:648] HybridClock initialized: now 1786291922807085 us; error 31 us; skew 500 ppm
I20260809 16:12:02.808238 32526 webserver.cc:460] Webserver started at http://127.16.224.190:46083/ using document root <none> and password file <none>
I20260809 16:12:02.808429 32526 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:02.808470 32526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:02.808563 32526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:02.809566 32526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "8e2931401ef34e9dbe908adc9535578a"
format_stamp: "Formatted at 2026-08-09 16:12:02 on dist-test-slave-fgb3"
server_key: "3aeec08acd2194ea53208ef9d1f2764e"
server_key_iv: "eb79c0851b5c4095343d4c9d55b7e442"
server_key_version: "encryptionkey@0"
I20260809 16:12:02.809873 32526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "8e2931401ef34e9dbe908adc9535578a"
format_stamp: "Formatted at 2026-08-09 16:12:02 on dist-test-slave-fgb3"
server_key: "3aeec08acd2194ea53208ef9d1f2764e"
server_key_iv: "eb79c0851b5c4095343d4c9d55b7e442"
server_key_version: "encryptionkey@0"
I20260809 16:12:02.811107 32526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:12:02.811743 32547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:02.811901 32526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:12:02.811985 32526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "8e2931401ef34e9dbe908adc9535578a"
format_stamp: "Formatted at 2026-08-09 16:12:02 on dist-test-slave-fgb3"
server_key: "3aeec08acd2194ea53208ef9d1f2764e"
server_key_iv: "eb79c0851b5c4095343d4c9d55b7e442"
server_key_version: "encryptionkey@0"
I20260809 16:12:02.812036 32526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:02.846218 32526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:02.846589 32526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:02.846709 32526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:02.850417 32526 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:41941
I20260809 16:12:02.850456 32636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:41941 every 8 connection(s)
I20260809 16:12:02.850775 32526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:12:02.851433 32637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:02.854097 32637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a: Bootstrap starting.
I20260809 16:12:02.854758 32637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:02.855036 32637 log.cc:824] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:02.855688 32637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a: No bootstrap required, opened a new log
I20260809 16:12:02.857224 32637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2931401ef34e9dbe908adc9535578a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41941 } }
I20260809 16:12:02.857363 32637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:02.857385 32637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2931401ef34e9dbe908adc9535578a, State: Initialized, Role: FOLLOWER
I20260809 16:12:02.857535 32637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2931401ef34e9dbe908adc9535578a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41941 } }
I20260809 16:12:02.857602 32637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:12:02.857625 32637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:12:02.857671 32637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:02.858177 32637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2931401ef34e9dbe908adc9535578a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41941 } }
I20260809 16:12:02.858273 32637 leader_election.cc:302] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e2931401ef34e9dbe908adc9535578a; no voters:
I20260809 16:12:02.858455 32637 leader_election.cc:288] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:12:02.858496 32640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:12:02.858625 32640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [term 1 LEADER]: Becoming Leader. State: Replica: 8e2931401ef34e9dbe908adc9535578a, State: Running, Role: LEADER
I20260809 16:12:02.858736 32637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:12:02.858783 32640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2931401ef34e9dbe908adc9535578a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41941 } }
I20260809 16:12:02.859186 32641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e2931401ef34e9dbe908adc9535578a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2931401ef34e9dbe908adc9535578a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41941 } } }
I20260809 16:12:02.859273 32641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:02.859301 32644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e2931401ef34e9dbe908adc9535578a. Latest consensus state: current_term: 1 leader_uuid: "8e2931401ef34e9dbe908adc9535578a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2931401ef34e9dbe908adc9535578a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41941 } } }
I20260809 16:12:02.859381 32644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:02.859700 32651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:12:02.860178 32651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:12:02.860296 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 32526
I20260809 16:12:02.860383 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:12:02.860575 17282 external_mini_cluster.cc:1428] Setting key 10c4eaa0e70bbec0790aa4d3fbd85c64
I20260809 16:12:02.861887 32651 catalog_manager.cc:1347] Generated new cluster ID: 09cd1eca41e2421eadd7576cdad8d528
I20260809 16:12:02.861943 32651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:12:02.866649 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:02.861019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.16.224.190:41941)
0809 16:12:02.861204 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:02.861214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:02.861243 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:12:02.861373 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:02.861381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:02.861449 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:02.861576 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:02.861903 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.863285 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.864486 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:02.865122 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:02.865232 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:02.865934 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:02.865938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:02.865939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:02.866148 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:02.866190 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:02.866205 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:02.866267 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:02.866374 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:12:02.866450 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260809 16:12:02.867378 32651 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:12:02.867832 32651 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:12:02.883054 32651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e2931401ef34e9dbe908adc9535578a: Generated new TSK 0
I20260809 16:12:02.883253 32651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:12:02.884773 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41941
--builtin_ntp_servers=127.16.224.148:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 16:12:02.986084 32670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:02.986292 32670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:02.986334 32670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:02.987514 32670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:02.987553 32670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:02.987577 32670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:12:02.988905 32670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:02.989146 32670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:02.989449 32670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:02.994355 32685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:02.994355 32682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:02.994478 32683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:02.994917 32670 server_base.cc:1034] running on GCE node
I20260809 16:12:02.997565 32670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:02.997853 32670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:02.999022 32670 hybrid_clock.cc:648] HybridClock initialized: now 1786291922998992 us; error 44 us; skew 500 ppm
I20260809 16:12:03.000497 32670 webserver.cc:460] Webserver started at http://127.16.224.129:35999/ using document root <none> and password file <none>
I20260809 16:12:03.000771 32670 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:03.000826 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:03.000957 32670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:03.002385 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "36b77173f4d44d9fb5604a77e0bdc36e"
format_stamp: "Formatted at 2026-08-09 16:12:02 on dist-test-slave-fgb3"
server_key: "c219ea568fccb0a9ddb430c0ddbcee92"
server_key_iv: "c8ab6cec143d0f40f12cb86829cad2c8"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.002844 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "36b77173f4d44d9fb5604a77e0bdc36e"
format_stamp: "Formatted at 2026-08-09 16:12:02 on dist-test-slave-fgb3"
server_key: "c219ea568fccb0a9ddb430c0ddbcee92"
server_key_iv: "c8ab6cec143d0f40f12cb86829cad2c8"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.004508 32670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:12:03.005322 32694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.005486 32670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.005571 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "36b77173f4d44d9fb5604a77e0bdc36e"
format_stamp: "Formatted at 2026-08-09 16:12:02 on dist-test-slave-fgb3"
server_key: "c219ea568fccb0a9ddb430c0ddbcee92"
server_key_iv: "c8ab6cec143d0f40f12cb86829cad2c8"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.005630 32670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:03.016805 32670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:03.017059 32670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:03.017158 32670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:03.017340 32670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:03.017617 32670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:03.017648 32670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.017669 32670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:03.017685 32670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.023242 32670 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:42425
I20260809 16:12:03.023288 408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:42425 every 8 connection(s)
I20260809 16:12:03.023618 32670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:12:03.028700 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.024339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:55655 (local address 127.16.224.190:41941)
0809 16:12:03.024391 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.024393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.024629 (+ 236us) server_negotiation.cc:407] Connection header received
0809 16:12:03.025149 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.025150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.025165 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.025219 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.025877 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.026427 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.027392 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.027597 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.027654 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:03.028431 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:03.028433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:03.028433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:03.028441 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:03.028458 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:03.028463 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:03.028507 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:03.028587 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 16:12:03.028619 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:12:03.028892 413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.024311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41941 (local address 127.16.224.129:55655)
0809 16:12:03.024514 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:03.024549 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:03.025051 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:03.025288 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:03.025291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:03.025301 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:03.025788 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:03.025793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.026515 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.026516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:03.027305 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:03.027307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.027669 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.027670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:03.027793 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:03.028293 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:12:03.028294 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:03.028323 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:12:03.028348 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:03.028516 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:03.028522 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:03.028525 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:12:03.028585 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:12:03.028694 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260809 16:12:03.029371 409 heartbeater.cc:347] Connected to a master server at 127.16.224.190:41941
I20260809 16:12:03.029475 409 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:03.029659 409 heartbeater.cc:511] Master 127.16.224.190:41941 requested a full tablet report, sending...
I20260809 16:12:03.030157 32571 ts_manager.cc:194] Registered new tserver with Master: 36b77173f4d44d9fb5604a77e0bdc36e (127.16.224.129:42425)
I20260809 16:12:03.030797 32571 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.129:55655
I20260809 16:12:03.031040 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 32670
I20260809 16:12:03.031118 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:12:03.031298 17282 external_mini_cluster.cc:1428] Setting key e833c07ca5e69a83f79e1aeaf796c4b8
I20260809 16:12:03.032101 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41941
--builtin_ntp_servers=127.16.224.148:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 16:12:03.112936 414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:03.113127 414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:03.113155 414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:03.114282 414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:03.114336 414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:03.114370 414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:12:03.115723 414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:03.115959 414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:03.116225 414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:03.120877 426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.120878 422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.120944 414 server_base.cc:1034] running on GCE node
W20260809 16:12:03.120904 424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.121268 414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:03.121472 414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:03.122603 414 hybrid_clock.cc:648] HybridClock initialized: now 1786291923122587 us; error 32 us; skew 500 ppm
I20260809 16:12:03.123699 414 webserver.cc:460] Webserver started at http://127.16.224.130:40551/ using document root <none> and password file <none>
I20260809 16:12:03.123898 414 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:03.123935 414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:03.124032 414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:03.124963 414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "ebf591efbf674f2c9fb740e2809798a3"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "c9b093541a0e9745fbc6ea442934c3ad"
server_key_iv: "95035c83696dabc06531e52374126a2f"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.125281 414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "ebf591efbf674f2c9fb740e2809798a3"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "c9b093541a0e9745fbc6ea442934c3ad"
server_key_iv: "95035c83696dabc06531e52374126a2f"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.126413 414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:12:03.127058 436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.127195 414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.127277 414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "ebf591efbf674f2c9fb740e2809798a3"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "c9b093541a0e9745fbc6ea442934c3ad"
server_key_iv: "95035c83696dabc06531e52374126a2f"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.127331 414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:03.148833 414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:03.149116 414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:03.149217 414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:03.149411 414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:03.149701 414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:03.149731 414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.149760 414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:03.149772 414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.155120 414 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:39795
I20260809 16:12:03.155189 634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:39795 every 8 connection(s)
I20260809 16:12:03.155488 414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:12:03.157202 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 414
I20260809 16:12:03.157287 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:12:03.157506 17282 external_mini_cluster.cc:1428] Setting key e39ab97e3024bd6fd1ecc06e031ee987
I20260809 16:12:03.158573 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41941
--builtin_ntp_servers=127.16.224.148:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 16:12:03.161288 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.156183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:57133 (local address 127.16.224.190:41941)
0809 16:12:03.156235 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.156238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.156497 (+ 259us) server_negotiation.cc:407] Connection header received
0809 16:12:03.156984 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.156986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.157016 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.157066 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.157927 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.158582 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.159917 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.160106 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.160152 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:03.161034 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:03.161036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:03.161036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:03.161041 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:03.161060 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:03.161065 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:03.161115 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:03.161181 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:12:03.161208 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:12:03.161427 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.156172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41941 (local address 127.16.224.130:57133)
0809 16:12:03.156366 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:03.156402 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:03.156891 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:03.157160 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:03.157165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:03.157176 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:03.157831 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:03.157838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.158675 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.158676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:03.159837 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:03.159840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.160179 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.160180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:03.160342 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:03.160885 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:12:03.160885 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:03.160922 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:12:03.160945 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:03.161124 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:03.161127 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:03.161129 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:12:03.161170 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:12:03.161249 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260809 16:12:03.161871 635 heartbeater.cc:347] Connected to a master server at 127.16.224.190:41941
I20260809 16:12:03.161971 635 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:03.162159 635 heartbeater.cc:511] Master 127.16.224.190:41941 requested a full tablet report, sending...
I20260809 16:12:03.162537 32571 ts_manager.cc:194] Registered new tserver with Master: ebf591efbf674f2c9fb740e2809798a3 (127.16.224.130:39795)
I20260809 16:12:03.162894 32571 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.130:57133
W20260809 16:12:03.242683 641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:03.242888 641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:03.242916 641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:03.244073 641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:03.244115 641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:03.244148 641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:12:03.245496 641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:41941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:03.245731 641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:03.246035 641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:03.250967 649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.250976 650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.250974 653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.251283 641 server_base.cc:1034] running on GCE node
I20260809 16:12:03.251513 641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:03.251734 641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:03.252897 641 hybrid_clock.cc:648] HybridClock initialized: now 1786291923252882 us; error 30 us; skew 500 ppm
I20260809 16:12:03.254135 641 webserver.cc:460] Webserver started at http://127.16.224.131:42523/ using document root <none> and password file <none>
I20260809 16:12:03.254446 641 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:03.254511 641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:03.254650 641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:03.255728 641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "e3cf083acaca4d26930847bc2f6af76f"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "15e9d362573cfca8e37d54ef82d7a4ca"
server_key_iv: "1d1dee2d131532d83bf6f4d9710da891"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.256085 641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "e3cf083acaca4d26930847bc2f6af76f"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "15e9d362573cfca8e37d54ef82d7a4ca"
server_key_iv: "1d1dee2d131532d83bf6f4d9710da891"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.257376 641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:12:03.258069 663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.258224 641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:12:03.258299 641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "e3cf083acaca4d26930847bc2f6af76f"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "15e9d362573cfca8e37d54ef82d7a4ca"
server_key_iv: "1d1dee2d131532d83bf6f4d9710da891"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.258409 641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:03.267022 641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:03.267288 641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:03.267417 641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:03.267629 641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:03.267944 641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:03.267978 641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.267999 641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:03.268021 641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.273281 641 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:39311
I20260809 16:12:03.273325 849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:39311 every 8 connection(s)
I20260809 16:12:03.273638 641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:12:03.274791 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 641
I20260809 16:12:03.274878 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:12:03.275070 17282 external_mini_cluster.cc:1428] Setting key 3fc3f9487d16d682c9577ec5a8fd8ee0
I20260809 16:12:03.278630 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.274344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:55375 (local address 127.16.224.190:41941)
0809 16:12:03.274390 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.274392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.274626 (+ 234us) server_negotiation.cc:407] Connection header received
0809 16:12:03.275230 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.275232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.275248 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.275298 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.276007 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.276498 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.277409 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.277584 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.277626 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:03.278434 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:03.278435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:03.278435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:03.278440 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:03.278453 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:03.278456 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:03.278491 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:03.278548 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:12:03.278572 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:12:03.278853 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.274294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:41941 (local address 127.16.224.131:55375)
0809 16:12:03.274507 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:03.274541 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:03.275096 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:03.275429 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:03.275433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:03.275442 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:03.275939 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:03.275943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.276563 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.276564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:03.277347 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:03.277349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:03.277637 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:03.277638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:03.277744 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:03.278273 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:12:03.278273 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:03.278305 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:12:03.278360 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:03.278501 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:03.278504 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:03.278506 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:12:03.278551 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:12:03.278633 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260809 16:12:03.279253 850 heartbeater.cc:347] Connected to a master server at 127.16.224.190:41941
I20260809 16:12:03.279335 850 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:03.279476 850 heartbeater.cc:511] Master 127.16.224.190:41941 requested a full tablet report, sending...
I20260809 16:12:03.279815 32571 ts_manager.cc:194] Registered new tserver with Master: e3cf083acaca4d26930847bc2f6af76f (127.16.224.131:39311)
I20260809 16:12:03.280182 32571 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.131:55375
I20260809 16:12:03.286571 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:12:03.288306 32668 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:03.287810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.16.224.190:41941)
0809 16:12:03.287867 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.287868 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.287976 (+ 108us) server_negotiation.cc:407] Connection header received
0809 16:12:03.288040 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.288041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.288054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.288099 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.288239 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43312: BlockingRecv error: recv got EOF from 127.0.0.1:43312 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:12:03.288355 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 32670
I20260809 16:12:03.302414 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 414
I20260809 16:12:03.319978 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 641
I20260809 16:12:03.326447 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 32526
2026-08-09T16:12:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (643 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T16:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:12:03Z Disabled control of system clock
I20260809 16:12:03.359112 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:39785
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:36941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:39785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:12:03.440622 870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:03.440842 870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:03.440863 870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:03.441939 870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:12:03.441972 870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:03.441982 870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:03.441993 870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:12:03.442003 870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:12:03.443387 870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:39785
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:39785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:03.443598 870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:03.443850 870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:03.448640 877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.448635 883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.448642 878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.449019 870 server_base.cc:1034] running on GCE node
I20260809 16:12:03.449258 870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:03.449543 870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:03.450702 870 hybrid_clock.cc:648] HybridClock initialized: now 1786291923450682 us; error 36 us; skew 500 ppm
I20260809 16:12:03.451988 870 webserver.cc:460] Webserver started at http://127.16.224.190:44999/ using document root <none> and password file <none>
I20260809 16:12:03.452240 870 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:03.452319 870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:03.452456 870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:03.453552 870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "979b34086a474e2e9e61f9028fcab787"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "1f5f532b0eed9a87ba2cda721b5ac420"
server_key_iv: "527abbcccc3307d66cd475a26a2ecd8b"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.453923 870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "979b34086a474e2e9e61f9028fcab787"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "1f5f532b0eed9a87ba2cda721b5ac420"
server_key_iv: "527abbcccc3307d66cd475a26a2ecd8b"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.455363 870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:12:03.456084 891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.456238 870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:12:03.456351 870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "979b34086a474e2e9e61f9028fcab787"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "1f5f532b0eed9a87ba2cda721b5ac420"
server_key_iv: "527abbcccc3307d66cd475a26a2ecd8b"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.456418 870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:03.459838 870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:03.460101 870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:03.460227 870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:03.464375 870 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:39785
I20260809 16:12:03.464407 979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:39785 every 8 connection(s)
I20260809 16:12:03.464728 870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:12:03.465387 983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:03.468360 983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787: Bootstrap starting.
I20260809 16:12:03.468972 983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:03.469259 983 log.cc:824] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:03.469926 983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787: No bootstrap required, opened a new log
I20260809 16:12:03.471598 983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979b34086a474e2e9e61f9028fcab787" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39785 } }
I20260809 16:12:03.471745 983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:03.471766 983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979b34086a474e2e9e61f9028fcab787, State: Initialized, Role: FOLLOWER
I20260809 16:12:03.471911 983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979b34086a474e2e9e61f9028fcab787" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39785 } }
I20260809 16:12:03.471982 983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:12:03.472021 983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:12:03.472069 983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:03.472596 983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979b34086a474e2e9e61f9028fcab787" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39785 } }
I20260809 16:12:03.472692 983 leader_election.cc:302] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 979b34086a474e2e9e61f9028fcab787; no voters:
I20260809 16:12:03.472847 983 leader_election.cc:288] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:12:03.472891 987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:12:03.473047 987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [term 1 LEADER]: Becoming Leader. State: Replica: 979b34086a474e2e9e61f9028fcab787, State: Running, Role: LEADER
I20260809 16:12:03.473120 983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:12:03.473143 987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979b34086a474e2e9e61f9028fcab787" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39785 } }
I20260809 16:12:03.473349 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 870
I20260809 16:12:03.473460 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:12:03.473667 17282 external_mini_cluster.cc:1428] Setting key 3575790124c7b0ad9006f0583170ee0a
I20260809 16:12:03.473906 989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 979b34086a474e2e9e61f9028fcab787. Latest consensus state: current_term: 1 leader_uuid: "979b34086a474e2e9e61f9028fcab787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979b34086a474e2e9e61f9028fcab787" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39785 } } }
I20260809 16:12:03.473987 989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:03.474100 988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "979b34086a474e2e9e61f9028fcab787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979b34086a474e2e9e61f9028fcab787" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39785 } } }
I20260809 16:12:03.474172 988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787 [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:03.474206 998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:12:03.475059 998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:12:03.477380 998 catalog_manager.cc:1347] Generated new cluster ID: 1c6b1f3530714764ac3ba653f3f38e8d
I20260809 16:12:03.477437 998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:12:03.477707 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.474355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.16.224.190:39785)
0809 16:12:03.476110 (+ 1755us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.476119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.476153 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:12:03.476271 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.476277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.476310 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.476449 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.477145 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:03.477149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:03.477149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:03.477153 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:03.477193 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:03.477203 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:03.477257 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:03.477477 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 16:12:03.477539 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1679,"thread_start_us":102,"threads_started":1}
I20260809 16:12:03.483786 998 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:12:03.483938 998 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:12:03.486891 998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 979b34086a474e2e9e61f9028fcab787: Generated new TSK 0
I20260809 16:12:03.487071 998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:12:03.493988 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39785
--builtin_ntp_servers=127.16.224.148:36941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:12:03.577488 1021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:03.577697 1021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:03.577718 1021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:03.578955 1021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:03.578999 1021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:03.579025 1021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:12:03.580433 1021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:03.580667 1021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:03.580971 1021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:03.585831 1034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.585886 1029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.585964 1021 server_base.cc:1034] running on GCE node
W20260809 16:12:03.585839 1027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.586458 1021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:03.586699 1021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:03.587863 1021 hybrid_clock.cc:648] HybridClock initialized: now 1786291923587845 us; error 34 us; skew 500 ppm
I20260809 16:12:03.589187 1021 webserver.cc:460] Webserver started at http://127.16.224.129:35309/ using document root <none> and password file <none>
I20260809 16:12:03.589411 1021 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:03.589460 1021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:03.589592 1021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:03.590770 1021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "51ab7b4179d548d4a4bdb3b74bc0b367"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "fd1746b52ef81db035e7c66d296fb907"
server_key_iv: "a76d08cf6dcb22b1b5a16e7966574e4b"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.591131 1021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "51ab7b4179d548d4a4bdb3b74bc0b367"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "fd1746b52ef81db035e7c66d296fb907"
server_key_iv: "a76d08cf6dcb22b1b5a16e7966574e4b"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.592407 1021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:12:03.593107 1043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.593264 1021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:12:03.593366 1021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "51ab7b4179d548d4a4bdb3b74bc0b367"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "fd1746b52ef81db035e7c66d296fb907"
server_key_iv: "a76d08cf6dcb22b1b5a16e7966574e4b"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.593441 1021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:03.597874 1021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:03.598205 1021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:03.598380 1021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:03.598632 1021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:03.598999 1021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:03.599045 1021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.599079 1021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:03.599102 1021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.605144 1021 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:36007
I20260809 16:12:03.605211 1220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:36007 every 8 connection(s)
I20260809 16:12:03.605507 1021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:12:03.608424 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.606200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:59061 (local address 127.16.224.190:39785)
0809 16:12:03.606242 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.606244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.606533 (+ 289us) server_negotiation.cc:407] Connection header received
0809 16:12:03.607044 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.607046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.607055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.607105 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.608189 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:03.608191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:03.608192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:03.608196 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:03.608214 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:03.608218 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:03.608271 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:03.608338 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:12:03.608353 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:12:03.608637 1225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.606188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39785 (local address 127.16.224.129:59061)
0809 16:12:03.606416 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:03.606451 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:03.606948 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:03.607192 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:03.607197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:03.607207 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:03.608041 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:12:03.608042 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:03.608071 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:12:03.608102 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:03.608273 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:03.608279 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:03.608282 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:12:03.608350 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 16:12:03.608425 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260809 16:12:03.609082 1222 heartbeater.cc:347] Connected to a master server at 127.16.224.190:39785
I20260809 16:12:03.609161 1222 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:03.609303 1222 heartbeater.cc:511] Master 127.16.224.190:39785 requested a full tablet report, sending...
I20260809 16:12:03.609776 921 ts_manager.cc:194] Registered new tserver with Master: 51ab7b4179d548d4a4bdb3b74bc0b367 (127.16.224.129:36007)
I20260809 16:12:03.610277 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 1021
I20260809 16:12:03.610390 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:12:03.610584 17282 external_mini_cluster.cc:1428] Setting key d73d6c9f04d2379a1fcdec470345932d
I20260809 16:12:03.611339 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39785
--builtin_ntp_servers=127.16.224.148:36941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:12:03.694667 1226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:03.694866 1226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:03.694887 1226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:03.695996 1226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:03.696031 1226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:03.696054 1226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:12:03.697468 1226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:03.697678 1226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:03.697932 1226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:03.702752 1235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.702872 1226 server_base.cc:1034] running on GCE node
W20260809 16:12:03.702752 1240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.703745 1236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.703963 1226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:03.704187 1226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:03.705324 1226 hybrid_clock.cc:648] HybridClock initialized: now 1786291923705311 us; error 30 us; skew 500 ppm
I20260809 16:12:03.706607 1226 webserver.cc:460] Webserver started at http://127.16.224.130:34305/ using document root <none> and password file <none>
I20260809 16:12:03.706842 1226 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:03.706892 1226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:03.707023 1226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:03.708081 1226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "2171faedaec942b18285d193061ee90b"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "ba99b752252742beb22613ded73e16db"
server_key_iv: "647f813f6e3efc4aa8915e353a7ee496"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.708432 1226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "2171faedaec942b18285d193061ee90b"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "ba99b752252742beb22613ded73e16db"
server_key_iv: "647f813f6e3efc4aa8915e353a7ee496"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.709630 1226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:12:03.710261 1251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.710423 1226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:12:03.710510 1226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "2171faedaec942b18285d193061ee90b"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "ba99b752252742beb22613ded73e16db"
server_key_iv: "647f813f6e3efc4aa8915e353a7ee496"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.710578 1226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:03.713867 1226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:03.714119 1226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:03.714236 1226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:03.714453 1226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:03.714751 1226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:03.714784 1226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.714810 1226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:03.714834 1226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.720378 1226 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:33025
I20260809 16:12:03.720418 1438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:33025 every 8 connection(s)
I20260809 16:12:03.720736 1226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:12:03.723534 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.721418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:58765 (local address 127.16.224.190:39785)
0809 16:12:03.721473 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.721474 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.721688 (+ 214us) server_negotiation.cc:407] Connection header received
0809 16:12:03.722144 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.722145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.722150 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.722213 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.723350 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:03.723351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:03.723351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:03.723353 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:03.723366 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:03.723369 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:03.723421 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:03.723467 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 16:12:03.723478 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:12:03.723702 1441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.721389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39785 (local address 127.16.224.130:58765)
0809 16:12:03.721571 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:03.721612 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:03.722068 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:03.722268 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:03.722271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:03.722279 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:03.723215 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:12:03.723215 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:03.723244 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:12:03.723274 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:03.723420 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:03.723422 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:03.723425 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:12:03.723474 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:12:03.723520 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260809 16:12:03.724154 1439 heartbeater.cc:347] Connected to a master server at 127.16.224.190:39785
I20260809 16:12:03.724242 1439 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:03.724388 1439 heartbeater.cc:511] Master 127.16.224.190:39785 requested a full tablet report, sending...
I20260809 16:12:03.724761 921 ts_manager.cc:194] Registered new tserver with Master: 2171faedaec942b18285d193061ee90b (127.16.224.130:33025)
I20260809 16:12:03.726900 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 1226
I20260809 16:12:03.726990 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:12:03.727169 17282 external_mini_cluster.cc:1428] Setting key 90b39d780f0d6894980c39f4fd143cf1
I20260809 16:12:03.727983 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39785
--builtin_ntp_servers=127.16.224.148:36941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:12:03.810993 1446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:03.811188 1446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:03.811208 1446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:03.812268 1446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:03.812304 1446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:03.812327 1446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:12:03.813673 1446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:03.813851 1446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:03.814110 1446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:12:03.818785 1446 server_base.cc:1034] running on GCE node
W20260809 16:12:03.818827 1458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.818768 1459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:03.818769 1461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:03.819205 1446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:03.819427 1446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:03.820568 1446 hybrid_clock.cc:648] HybridClock initialized: now 1786291923820552 us; error 34 us; skew 500 ppm
I20260809 16:12:03.821768 1446 webserver.cc:460] Webserver started at http://127.16.224.131:42765/ using document root <none> and password file <none>
I20260809 16:12:03.821966 1446 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:03.822012 1446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:03.822136 1446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:03.823220 1446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "c08fa8308f534078889838717729a8db"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "b8423a62af3cc49a75b6d5487a266002"
server_key_iv: "92ad2ab9630e1d94ed7ab610db217535"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.823554 1446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "c08fa8308f534078889838717729a8db"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "b8423a62af3cc49a75b6d5487a266002"
server_key_iv: "92ad2ab9630e1d94ed7ab610db217535"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.824718 1446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:12:03.825330 1473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.825484 1446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:12:03.825562 1446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "c08fa8308f534078889838717729a8db"
format_stamp: "Formatted at 2026-08-09 16:12:03 on dist-test-slave-fgb3"
server_key: "b8423a62af3cc49a75b6d5487a266002"
server_key_iv: "92ad2ab9630e1d94ed7ab610db217535"
server_key_version: "encryptionkey@0"
I20260809 16:12:03.825626 1446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:03.828792 1446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:03.829011 1446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:03.829125 1446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:03.829317 1446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:03.829602 1446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:03.829633 1446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.829663 1446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:03.829684 1446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:03.834939 1446 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:36129
I20260809 16:12:03.835000 1660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:36129 every 8 connection(s)
I20260809 16:12:03.835250 1446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:12:03.838153 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.835919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:53863 (local address 127.16.224.190:39785)
0809 16:12:03.835970 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.835972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.836182 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:12:03.836684 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.836686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.836694 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.836748 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.837893 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:03.837895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:03.837896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:12:03.837901 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:03.837919 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:12:03.837924 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:03.837972 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:03.838063 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:12:03.838080 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:12:03.838406 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:03.835893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39785 (local address 127.16.224.131:53863)
0809 16:12:03.836069 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:03.836102 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:03.836576 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:03.836818 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:03.836822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:03.836833 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:03.837731 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:12:03.837731 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:03.837761 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:12:03.837792 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:03.837977 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:03.837982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:03.837985 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:12:03.838069 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:12:03.838132 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260809 16:12:03.838855 1661 heartbeater.cc:347] Connected to a master server at 127.16.224.190:39785
I20260809 16:12:03.838943 1661 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:03.839068 1661 heartbeater.cc:511] Master 127.16.224.190:39785 requested a full tablet report, sending...
I20260809 16:12:03.839398 921 ts_manager.cc:194] Registered new tserver with Master: c08fa8308f534078889838717729a8db (127.16.224.131:36129)
I20260809 16:12:03.843140 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 1446
I20260809 16:12:03.843211 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:12:03.843410 17282 external_mini_cluster.cc:1428] Setting key 926810488516eeb05f9cff62500c4a28
I20260809 16:12:03.844189 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:12:03.845968 1005 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:03.845476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.16.224.190:39785)
0809 16:12:03.845531 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:03.845533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:03.845668 (+ 135us) server_negotiation.cc:407] Connection header received
0809 16:12:03.845713 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:03.845714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:03.845720 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:03.845766 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:03.845898 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48016: BlockingRecv error: recv got EOF from 127.0.0.1:48016 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:12:03.846021 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 1021
I20260809 16:12:03.863533 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 1226
I20260809 16:12:03.875254 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 1446
I20260809 16:12:03.881385 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 870
2026-08-09T16:12:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (554 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:12:03 dist-test-slave-fgb3 krb5kdc[1679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:12:03 dist-test-slave-fgb3 krb5kdc[1679](info): set up 2 sockets
Aug 09 16:12:03 dist-test-slave-fgb3 krb5kdc[1679](info): commencing operation
krb5kdc: starting...
W20260809 16:12:05.037748 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:12:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:12:05.150086 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46563
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:46563
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:05.230618 1696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:05.230827 1696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:05.230859 1696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:05.231980 1696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:12:05.232020 1696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:05.232038 1696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:05.232053 1696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:12:05.232067 1696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:12:05.233511 1696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:46563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:05.233762 1696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:05.233971 1696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:05.238610 1707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.238662 1704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.238610 1705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:05.238955 1696 server_base.cc:1034] running on GCE node
I20260809 16:12:05.239200 1696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:05.239485 1696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:05.240646 1696 hybrid_clock.cc:648] HybridClock initialized: now 1786291925240624 us; error 38 us; skew 500 ppm
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:05.243244 1696 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:12:05.243662 1696 webserver.cc:460] Webserver started at http://127.16.224.190:38375/ using document root <none> and password file <none>
I20260809 16:12:05.243862 1696 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:05.243906 1696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:05.244099 1696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:05.245239 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "24e578d57f1d4288a9a503ad8a7c8fd9"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "d2571c10175b1b0bcbc3d8658a5b65fb"
server_key_iv: "a6d58f03d5d06bf293d485b56e276727"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.245596 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "24e578d57f1d4288a9a503ad8a7c8fd9"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "d2571c10175b1b0bcbc3d8658a5b65fb"
server_key_iv: "a6d58f03d5d06bf293d485b56e276727"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.246919 1696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:12:05.247648 1716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.247790 1696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:12:05.247879 1696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "24e578d57f1d4288a9a503ad8a7c8fd9"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "d2571c10175b1b0bcbc3d8658a5b65fb"
server_key_iv: "a6d58f03d5d06bf293d485b56e276727"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.247937 1696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:05.260879 1696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:05.262627 1696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:05.262755 1696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:05.266618 1696 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:46563
I20260809 16:12:05.266651 1803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:46563 every 8 connection(s)
I20260809 16:12:05.266980 1696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:12:05.268021 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:05.270488 1804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9: Bootstrap starting.
I20260809 16:12:05.271085 1804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:05.271353 1804 log.cc:824] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:05.272073 1804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9: No bootstrap required, opened a new log
I20260809 16:12:05.273826 1804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46563 } }
I20260809 16:12:05.273974 1804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:05.273995 1804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e578d57f1d4288a9a503ad8a7c8fd9, State: Initialized, Role: FOLLOWER
I20260809 16:12:05.274143 1804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46563 } }
I20260809 16:12:05.274206 1804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:12:05.274238 1804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:12:05.274284 1804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:05.274857 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 1696
I20260809 16:12:05.274964 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:12:05.275167 17282 external_mini_cluster.cc:1428] Setting key f87d363a3d713121e1e9f24fa0714fd1
I20260809 16:12:05.275312 1804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46563 } }
I20260809 16:12:05.275427 1804 leader_election.cc:302] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24e578d57f1d4288a9a503ad8a7c8fd9; no voters:
I20260809 16:12:05.275596 1804 leader_election.cc:288] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:12:05.275645 1808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:12:05.275779 1808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [term 1 LEADER]: Becoming Leader. State: Replica: 24e578d57f1d4288a9a503ad8a7c8fd9, State: Running, Role: LEADER
I20260809 16:12:05.275864 1808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46563 } }
I20260809 16:12:05.275894 1804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:12:05.276223 1811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46563 } } }
I20260809 16:12:05.276309 1811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:05.276573 1817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24e578d57f1d4288a9a503ad8a7c8fd9. Latest consensus state: current_term: 1 leader_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e578d57f1d4288a9a503ad8a7c8fd9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46563 } } }
I20260809 16:12:05.276751 1817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:05.276947 1824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:12:05.277549 1824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:12:05.279237 1824 catalog_manager.cc:1347] Generated new cluster ID: a79cbb13dba84cd9a82b8527cb1b8ad2
I20260809 16:12:05.279290 1824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:05.287261 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.275650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.16.224.190:46563)
0809 16:12:05.275943 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.275953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.276350 (+ 397us) server_negotiation.cc:407] Connection header received
0809 16:12:05.276591 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.276601 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.276664 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.276781 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:05.277727 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.279017 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.280203 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.280851 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.280981 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.283681 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:05.283685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:05.283686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:05.283698 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:05.285592 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.286001 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.286004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.286004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.286119 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.286334 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.286335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.286335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.286635 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:05.286787 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.286902 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.287038 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
I20260809 16:12:05.290614 1824 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:12:05.291100 1824 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:12:05.296437 1824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9: Generated new TSK 0
I20260809 16:12:05.296604 1824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:12:05.323253 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46563
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:05.404584 1842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:05.404779 1842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:05.404800 1842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:05.405864 1842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:05.405896 1842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:05.405920 1842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:12:05.407366 1842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:05.407578 1842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:05.407763 1842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:05.412464 1848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.412465 1851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.412473 1849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:05.412757 1842 server_base.cc:1034] running on GCE node
I20260809 16:12:05.412956 1842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:05.413160 1842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:05.414345 1842 hybrid_clock.cc:648] HybridClock initialized: now 1786291925414295 us; error 34 us; skew 500 ppm
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:05.416720 1842 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:12:05.417138 1842 webserver.cc:460] Webserver started at http://127.16.224.129:33123/ using document root <none> and password file <none>
I20260809 16:12:05.417346 1842 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:05.417389 1842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:05.417517 1842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:05.418630 1842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "c8ac0cf30cab4087b0585b3c2e77a2da"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "b57a903c33d5a3aa74a2a4d33ede1a41"
server_key_iv: "3a230e7dbea0373b983ef5ef08067e9e"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.418962 1842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "c8ac0cf30cab4087b0585b3c2e77a2da"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "b57a903c33d5a3aa74a2a4d33ede1a41"
server_key_iv: "3a230e7dbea0373b983ef5ef08067e9e"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.420192 1842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:12:05.420842 1858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.420986 1842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:12:05.421072 1842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "c8ac0cf30cab4087b0585b3c2e77a2da"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "b57a903c33d5a3aa74a2a4d33ede1a41"
server_key_iv: "3a230e7dbea0373b983ef5ef08067e9e"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.421130 1842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:05.442240 1842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:05.444085 1842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:05.444199 1842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:05.444407 1842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:05.444716 1842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:05.444752 1842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.444782 1842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:05.444808 1842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.450428 1842 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:46759
I20260809 16:12:05.450491 2031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:46759 every 8 connection(s)
I20260809 16:12:05.450794 1842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:05.458240 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 1842
I20260809 16:12:05.458350 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:12:05.458554 17282 external_mini_cluster.cc:1428] Setting key 9f50ba1619ff89805e888ef914f4306b
I20260809 16:12:05.459730 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.451471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:57297 (local address 127.16.224.190:46563)
0809 16:12:05.451513 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.451515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.451742 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:12:05.452258 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.452259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.452277 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.452329 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:05.453262 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.453770 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.454798 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.455015 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.455073 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.456927 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:05.456928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:05.456928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:05.456936 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:05.458275 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.458734 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.458736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.458736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.458801 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.459105 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.459106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.459106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.459236 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:05.459287 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.459594 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.459665 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:12:05.459956 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.451443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46563 (local address 127.16.224.129:57297)
0809 16:12:05.451628 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.451667 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.452162 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.452407 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.452411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.452976 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:05.453167 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.453172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.453860 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.453861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.454721 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.454724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.455087 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.455087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.455203 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.455595 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:05.455603 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:05.456841 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:05.458387 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:05.458397 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:05.458399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:05.458666 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:05.458882 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:05.458883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:05.458883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:05.459041 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:05.459283 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:05.459291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:05.459459 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.459571 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.459731 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260809 16:12:05.460624 2032 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46563
I20260809 16:12:05.460729 2032 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:05.460918 2032 heartbeater.cc:511] Master 127.16.224.190:46563 requested a full tablet report, sending...
I20260809 16:12:05.461555 1741 ts_manager.cc:194] Registered new tserver with Master: c8ac0cf30cab4087b0585b3c2e77a2da (127.16.224.129:46759)
I20260809 16:12:05.462445 1741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:57297
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:12:05.482307 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46563
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:05.565124 2038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:05.565339 2038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:05.565361 2038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:05.566596 2038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:05.566642 2038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:05.566669 2038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:12:05.568244 2038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:05.568497 2038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:05.568734 2038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:05.573829 2052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.573856 2051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.573901 2054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:05.574028 2038 server_base.cc:1034] running on GCE node
I20260809 16:12:05.574283 2038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:05.574563 2038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:05.575717 2038 hybrid_clock.cc:648] HybridClock initialized: now 1786291925575702 us; error 25 us; skew 500 ppm
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:05.578423 2038 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:12:05.578804 2038 webserver.cc:460] Webserver started at http://127.16.224.130:44547/ using document root <none> and password file <none>
I20260809 16:12:05.578989 2038 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:05.579026 2038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:05.579119 2038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:05.580114 2038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "a31ba78774054007b38faaa317f9e226"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "839f4f68214e2b4d6395553ba66bbd43"
server_key_iv: "cf1ec2e42483f9d3a92d0d0e0816bdef"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.580410 2038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "a31ba78774054007b38faaa317f9e226"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "839f4f68214e2b4d6395553ba66bbd43"
server_key_iv: "cf1ec2e42483f9d3a92d0d0e0816bdef"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.581573 2038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:12:05.582214 2067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.582393 2038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:12:05.582480 2038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "a31ba78774054007b38faaa317f9e226"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "839f4f68214e2b4d6395553ba66bbd43"
server_key_iv: "cf1ec2e42483f9d3a92d0d0e0816bdef"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.582547 2038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:05.591274 2038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:05.592913 2038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:05.593022 2038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:05.593194 2038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:05.593510 2038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:05.593554 2038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.593577 2038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:05.593600 2038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.599164 2038 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:37561
I20260809 16:12:05.599236 2239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:37561 every 8 connection(s)
I20260809 16:12:05.599519 2038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:05.607637 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 2038
I20260809 16:12:05.607743 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:12:05.607919 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.600234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:46455 (local address 127.16.224.190:46563)
0809 16:12:05.600289 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.600292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.600475 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:12:05.600954 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.600955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.600966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.601019 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:05.601916 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.602470 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.603403 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.603590 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.603632 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.605594 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:05.605596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:05.605596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:05.605602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:05.606725 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.607083 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.607083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.607084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.607128 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.607298 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.607298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.607299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.607395 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:05.607437 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.607771 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.607852 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:12:05.607975 17282 external_mini_cluster.cc:1428] Setting key a9b565420b64016749bf7f118c419769
I20260809 16:12:05.608127 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.600206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46563 (local address 127.16.224.130:46455)
0809 16:12:05.600363 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.600397 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.600878 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.601086 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.601089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.601654 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:05.601847 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.601852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.602541 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.602542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.603339 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.603341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.603641 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.603642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.603742 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.604209 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:05.604218 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:05.605504 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:05.606801 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:05.606807 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:05.606809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:05.607031 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:05.607170 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:05.607171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:05.607171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:05.607257 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:05.607452 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:05.607458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:05.607626 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.607738 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.607900 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260809 16:12:05.608582 2240 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46563
I20260809 16:12:05.608688 2240 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:05.608882 2240 heartbeater.cc:511] Master 127.16.224.190:46563 requested a full tablet report, sending...
I20260809 16:12:05.609301 1741 ts_manager.cc:194] Registered new tserver with Master: a31ba78774054007b38faaa317f9e226 (127.16.224.130:37561)
I20260809 16:12:05.609638 1741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:46455
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:12:05.630919 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46563
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:05.715493 2247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:05.715684 2247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:05.715706 2247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:05.716813 2247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:05.716850 2247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:05.716873 2247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:12:05.718350 2247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:05.718540 2247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:05.718722 2247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:05.723352 2255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.723410 2254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:05.723390 2257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:05.723489 2247 server_base.cc:1034] running on GCE node
I20260809 16:12:05.723949 2247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:05.724187 2247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:05.725324 2247 hybrid_clock.cc:648] HybridClock initialized: now 1786291925725311 us; error 31 us; skew 500 ppm
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:05.727717 2247 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:12:05.728128 2247 webserver.cc:460] Webserver started at http://127.16.224.131:42061/ using document root <none> and password file <none>
I20260809 16:12:05.728336 2247 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:05.728384 2247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:05.728513 2247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:12:05.729557 2247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "002f599631b14da9911b14e542d9a9af"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "f91f4a54bc1186fb3aa725712d7c73a2"
server_key_iv: "c645d254a6fe11a0f8fa15f9cfc0657b"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.729895 2247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "002f599631b14da9911b14e542d9a9af"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "f91f4a54bc1186fb3aa725712d7c73a2"
server_key_iv: "c645d254a6fe11a0f8fa15f9cfc0657b"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.731184 2247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:12:05.731820 2267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.731967 2247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:12:05.732054 2247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "002f599631b14da9911b14e542d9a9af"
format_stamp: "Formatted at 2026-08-09 16:12:05 on dist-test-slave-fgb3"
server_key: "f91f4a54bc1186fb3aa725712d7c73a2"
server_key_iv: "c645d254a6fe11a0f8fa15f9cfc0657b"
server_key_version: "encryptionkey@0"
I20260809 16:12:05.732112 2247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:05.741755 2247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:05.743566 2247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:05.743693 2247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:05.743892 2247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:05.744170 2247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:05.744213 2247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.744241 2247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:05.744258 2247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:05.749719 2247 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:45009
I20260809 16:12:05.749778 2449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:45009 every 8 connection(s)
I20260809 16:12:05.750062 2247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:12:05.756299 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 2247
I20260809 16:12:05.756402 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:12:05.756589 17282 external_mini_cluster.cc:1428] Setting key d335607e963bacd1108d0f5b07565988
Aug 09 16:12:05 dist-test-slave-fgb3 krb5kdc[1679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291925, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:05.759310 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.750793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:55301 (local address 127.16.224.190:46563)
0809 16:12:05.750839 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.750841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.751049 (+ 208us) server_negotiation.cc:407] Connection header received
0809 16:12:05.751561 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.751562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.751572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.751626 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:05.752489 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.752976 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.754151 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.754473 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.754545 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.757115 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:05.757116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:05.757117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:05.757124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:05.758259 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.758627 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.758627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.758628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.758675 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.758847 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.758847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.758848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.758943 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:05.758986 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.759186 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.759243 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:12:05.759454 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.750746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46563 (local address 127.16.224.131:55301)
0809 16:12:05.750902 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.750942 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.751479 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.751682 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.751686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.752235 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:05.752425 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.752429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.753034 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.753034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.754054 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.754057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.754567 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.754569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.754722 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.755324 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:05.755334 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:05.757030 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:05.758343 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:05.758350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:05.758352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:05.758570 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:05.758720 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:05.758721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:05.758721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:05.758807 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:05.758984 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:05.758987 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:05.759093 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.759168 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.759279 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260809 16:12:05.759848 2450 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46563
I20260809 16:12:05.759938 2450 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:05.760082 2450 heartbeater.cc:511] Master 127.16.224.190:46563 requested a full tablet report, sending...
I20260809 16:12:05.760404 1741 ts_manager.cc:194] Registered new tserver with Master: 002f599631b14da9911b14e542d9a9af (127.16.224.131:45009)
I20260809 16:12:05.760744 1741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:55301
I20260809 16:12:05.768204 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:12:05.774977 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.769452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.16.224.190:46563)
0809 16:12:05.769499 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.769501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.769603 (+ 102us) server_negotiation.cc:407] Connection header received
0809 16:12:05.769668 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.769669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.769679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.769721 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:05.770287 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.770822 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.771535 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.771740 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.771792 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.772877 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:05.772879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:05.772879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:05.772887 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:05.773883 (+ 996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.774197 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.774198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.774198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.774256 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:05.774442 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:05.774443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:05.774443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:05.774597 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:05.774687 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.774806 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.774885 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:12:05.776575 1741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:12:05.778553 1741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:12:05.784467 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.780640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:46759 (local address 127.0.0.1:33904)
0809 16:12:05.780810 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.780854 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.780922 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.781344 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.781345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.781347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:05.781412 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.781413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.782346 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.782347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.783317 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.783320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.784153 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.784154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.784237 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.784252 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.784326 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.784385 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":54,"thread_start_us":64,"threads_started":1}
I20260809 16:12:05.784492 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.780753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.16.224.129:46759)
0809 16:12:05.781168 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.781172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.781180 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:12:05.781212 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.781214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.781225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.781492 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:05.781572 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.782223 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.783413 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.784067 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.784143 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.784201 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.784347 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.784396 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":90,"threads_started":1}
I20260809 16:12:05.784807 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.780651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:45009 (local address 127.0.0.1:40966)
0809 16:12:05.781020 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.781029 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.781067 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.781309 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.781312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.781314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:05.781391 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.781394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.782291 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.782293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.783658 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.783661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.784572 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.784573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.784650 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.784653 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.784702 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.784741 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":5,"thread_start_us":51,"threads_started":1}
I20260809 16:12:05.784857 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.780947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.16.224.131:45009)
0809 16:12:05.781139 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.781144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.781155 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:12:05.781191 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.781193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.781206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.781283 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:05.781490 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.782200 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.783745 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.784413 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.784480 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.784534 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.784726 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.784774 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260809 16:12:05.785742 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.780839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:37561 (local address 127.0.0.1:42554)
0809 16:12:05.782350 (+ 1511us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.782359 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.782409 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.782566 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.782567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.782569 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:05.782610 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.782611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.783805 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.783806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.784666 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.784669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.785509 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.785510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.785570 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.785575 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.785638 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.785678 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1499,"mutex_wait_us":3,"thread_start_us":55,"threads_started":1}
I20260809 16:12:05.785614 2340 tablet_service.cc:1467] Processing CreateTablet for tablet e93e18717edd481abae1d4b363605d38 (DEFAULT_TABLE table=test-table [id=e762c34c4f8d4bd7b9c416608cd2ccf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:12:05.785768 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.780958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.16.224.130:37561)
0809 16:12:05.781425 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.781430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.782412 (+ 982us) server_negotiation.cc:407] Connection header received
0809 16:12:05.782468 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.782470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.782485 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.782564 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:05.782676 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.783728 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.784750 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.785423 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.785497 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.785547 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.785652 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.785691 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":86,"threads_started":1}
I20260809 16:12:05.785804 1935 tablet_service.cc:1467] Processing CreateTablet for tablet e93e18717edd481abae1d4b363605d38 (DEFAULT_TABLE table=test-table [id=e762c34c4f8d4bd7b9c416608cd2ccf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:12:05.786026 2340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93e18717edd481abae1d4b363605d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:05.786201 1935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93e18717edd481abae1d4b363605d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:05.786734 2142 tablet_service.cc:1467] Processing CreateTablet for tablet e93e18717edd481abae1d4b363605d38 (DEFAULT_TABLE table=test-table [id=e762c34c4f8d4bd7b9c416608cd2ccf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:12:05.787057 2142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93e18717edd481abae1d4b363605d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:05.788936 2468 tablet_bootstrap.cc:492] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da: Bootstrap starting.
I20260809 16:12:05.789561 2470 tablet_bootstrap.cc:492] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Bootstrap starting.
I20260809 16:12:05.789716 2468 tablet_bootstrap.cc:654] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:05.790136 2471 tablet_bootstrap.cc:492] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Bootstrap starting.
I20260809 16:12:05.790196 2470 tablet_bootstrap.cc:654] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:05.790484 2468 log.cc:824] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:05.790766 2471 tablet_bootstrap.cc:654] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:05.790809 2470 log.cc:824] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:05.791245 2468 tablet_bootstrap.cc:492] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da: No bootstrap required, opened a new log
I20260809 16:12:05.791319 2468 ts_tablet_manager.cc:1397] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:12:05.791347 2471 log.cc:824] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:05.791563 2470 tablet_bootstrap.cc:492] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: No bootstrap required, opened a new log
I20260809 16:12:05.791630 2470 ts_tablet_manager.cc:1397] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:12:05.791989 2471 tablet_bootstrap.cc:492] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: No bootstrap required, opened a new log
I20260809 16:12:05.792047 2471 ts_tablet_manager.cc:1397] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:12:05.793244 2468 raft_consensus.cc:348] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.793419 2468 raft_consensus.cc:374] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:05.793455 2468 raft_consensus.cc:729] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8ac0cf30cab4087b0585b3c2e77a2da, State: Initialized, Role: FOLLOWER
I20260809 16:12:05.793408 2470 raft_consensus.cc:348] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.793540 2470 raft_consensus.cc:374] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:05.793584 2470 raft_consensus.cc:729] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 002f599631b14da9911b14e542d9a9af, State: Initialized, Role: FOLLOWER
I20260809 16:12:05.793574 2468 consensus_queue.cc:260] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.793625 2471 raft_consensus.cc:348] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.793711 2470 consensus_queue.cc:260] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.793781 2471 raft_consensus.cc:374] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:05.793820 2471 raft_consensus.cc:729] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31ba78774054007b38faaa317f9e226, State: Initialized, Role: FOLLOWER
I20260809 16:12:05.793875 2468 ts_tablet_manager.cc:1428] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:12:05.793954 2470 ts_tablet_manager.cc:1428] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:12:05.793999 2032 heartbeater.cc:503] Master 127.16.224.190:46563 was elected leader, sending a full tablet report...
I20260809 16:12:05.794057 2450 heartbeater.cc:503] Master 127.16.224.190:46563 was elected leader, sending a full tablet report...
I20260809 16:12:05.793944 2471 consensus_queue.cc:260] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.794487 2240 heartbeater.cc:503] Master 127.16.224.190:46563 was elected leader, sending a full tablet report...
I20260809 16:12:05.794651 2471 ts_tablet_manager.cc:1428] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260809 16:12:05.850099 2243 tablet.cc:2369] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:12:05.913996 2482 raft_consensus.cc:482] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:12:05.914166 2482 raft_consensus.cc:504] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.914602 2482 leader_election.cc:288] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8ac0cf30cab4087b0585b3c2e77a2da (127.16.224.129:46759), 002f599631b14da9911b14e542d9a9af (127.16.224.131:45009)
I20260809 16:12:05.917248 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.914903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:45119 (local address 127.16.224.131:45009)
0809 16:12:05.914953 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.914955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.915033 (+ 78us) server_negotiation.cc:407] Connection header received
0809 16:12:05.915092 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.915093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.915103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.915147 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:05.915279 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.915740 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.916605 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.916958 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.917019 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.917029 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.917149 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.917182 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:12:05.917271 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.914892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:53843 (local address 127.16.224.129:46759)
0809 16:12:05.914938 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.914940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.914950 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:12:05.914979 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.914980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.915001 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.915050 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:05.915193 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.915668 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.916582 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.916959 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.917029 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.917037 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.917169 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.917202 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:12:05.917281 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.914772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:46759 (local address 127.16.224.130:53843)
0809 16:12:05.914828 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.914841 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.914904 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.915058 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.915060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.915061 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:05.915134 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.915136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.915739 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.915740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.916484 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.916486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.917028 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.917029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.917110 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.917113 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.917158 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.917198 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":1}
I20260809 16:12:05.917294 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.914772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:45009 (local address 127.16.224.130:45119)
0809 16:12:05.914949 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.914959 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:05.915030 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:05.915156 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:05.915157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:05.915158 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:05.915225 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.915226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.915813 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.915814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.916527 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:05.916529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.917032 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.917033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:05.917092 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.917095 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:12:05.917139 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:12:05.917177 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":16,"thread_start_us":99,"threads_started":1}
I20260809 16:12:05.917636 1962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93e18717edd481abae1d4b363605d38" candidate_uuid: "a31ba78774054007b38faaa317f9e226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" is_pre_election: true
I20260809 16:12:05.917626 2371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93e18717edd481abae1d4b363605d38" candidate_uuid: "a31ba78774054007b38faaa317f9e226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "002f599631b14da9911b14e542d9a9af" is_pre_election: true
I20260809 16:12:05.917788 2371 raft_consensus.cc:2454] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31ba78774054007b38faaa317f9e226 in term 0.
I20260809 16:12:05.917788 1962 raft_consensus.cc:2454] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31ba78774054007b38faaa317f9e226 in term 0.
I20260809 16:12:05.918049 2068 leader_election.cc:302] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31ba78774054007b38faaa317f9e226, c8ac0cf30cab4087b0585b3c2e77a2da; no voters:
I20260809 16:12:05.918203 2482 raft_consensus.cc:2781] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:12:05.918257 2482 raft_consensus.cc:482] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:12:05.918278 2482 raft_consensus.cc:3037] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:05.919016 2482 raft_consensus.cc:504] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.919134 2482 leader_election.cc:288] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [CANDIDATE]: Term 1 election: Requested vote from peers c8ac0cf30cab4087b0585b3c2e77a2da (127.16.224.129:46759), 002f599631b14da9911b14e542d9a9af (127.16.224.131:45009)
I20260809 16:12:05.919289 1962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93e18717edd481abae1d4b363605d38" candidate_uuid: "a31ba78774054007b38faaa317f9e226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da"
I20260809 16:12:05.919305 2371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93e18717edd481abae1d4b363605d38" candidate_uuid: "a31ba78774054007b38faaa317f9e226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "002f599631b14da9911b14e542d9a9af"
I20260809 16:12:05.919375 2371 raft_consensus.cc:3037] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:05.919376 1962 raft_consensus.cc:3037] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:05.920271 2371 raft_consensus.cc:2454] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31ba78774054007b38faaa317f9e226 in term 1.
I20260809 16:12:05.920271 1962 raft_consensus.cc:2454] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31ba78774054007b38faaa317f9e226 in term 1.
I20260809 16:12:05.920449 2068 leader_election.cc:302] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a31ba78774054007b38faaa317f9e226, c8ac0cf30cab4087b0585b3c2e77a2da; no voters:
I20260809 16:12:05.920527 2482 raft_consensus.cc:2781] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:12:05.920660 2482 raft_consensus.cc:686] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 1 LEADER]: Becoming Leader. State: Replica: a31ba78774054007b38faaa317f9e226, State: Running, Role: LEADER
I20260809 16:12:05.920749 2482 consensus_queue.cc:237] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } }
I20260809 16:12:05.921628 1740 catalog_manager.cc:5495] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 reported cstate change: term changed from 0 to 1, leader changed from <none> to a31ba78774054007b38faaa317f9e226 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "a31ba78774054007b38faaa317f9e226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31ba78774054007b38faaa317f9e226" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 37561 } health_report { overall_health: HEALTHY } } }
I20260809 16:12:05.931723 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.929468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.16.224.130:37561)
0809 16:12:05.929514 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:05.929516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:05.929524 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:12:05.929591 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:05.929592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:05.929611 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:05.929673 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:12:05.929794 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.930258 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.931012 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:05.931202 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:05.931254 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:05.931401 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:12:05.931505 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:12:05.931575 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:05.931610 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:12:05.931631 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:12:05.933377 2371 raft_consensus.cc:1261] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 1 FOLLOWER]: Refusing update from remote peer a31ba78774054007b38faaa317f9e226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:12:05.933377 1962 raft_consensus.cc:1261] T e93e18717edd481abae1d4b363605d38 P c8ac0cf30cab4087b0585b3c2e77a2da [term 1 FOLLOWER]: Refusing update from remote peer a31ba78774054007b38faaa317f9e226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:12:05.933668 2482 consensus_queue.cc:1035] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8ac0cf30cab4087b0585b3c2e77a2da" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:12:05.933809 2485 consensus_queue.cc:1035] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "002f599631b14da9911b14e542d9a9af" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:12:05.936002 2488 mvcc.cc:205] Tried to move back new op lower bound from 7316651728620736512 to 7316651728572047360. Current Snapshot: MvccSnapshot[applied={T|T < 7316651728620736512}]
I20260809 16:12:05.936332 2490 mvcc.cc:205] Tried to move back new op lower bound from 7316651728620736512 to 7316651728572047360. Current Snapshot: MvccSnapshot[applied={T|T < 7316651728620736512}]
I20260809 16:12:05.936445 2489 mvcc.cc:205] Tried to move back new op lower bound from 7316651728620736512 to 7316651728572047360. Current Snapshot: MvccSnapshot[applied={T|T < 7316651728620736512}]
I20260809 16:12:05.938405 1741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:12:05.938485 1741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:12:05.939213 1741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24e578d57f1d4288a9a503ad8a7c8fd9: Sending DeleteTablet for 3 replicas of tablet e93e18717edd481abae1d4b363605d38
I20260809 16:12:05.939548 2340 tablet_service.cc:1514] Processing DeleteTablet for tablet e93e18717edd481abae1d4b363605d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:12:05 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:40966
I20260809 16:12:05.939574 1935 tablet_service.cc:1514] Processing DeleteTablet for tablet e93e18717edd481abae1d4b363605d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:12:05 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:33904
I20260809 16:12:05.939615 2142 tablet_service.cc:1514] Processing DeleteTablet for tablet e93e18717edd481abae1d4b363605d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:12:05 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:42554
I20260809 16:12:05.939821 2503 tablet_replica.cc:330] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: stopping tablet replica
I20260809 16:12:05.939839 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 1842
I20260809 16:12:05.939913 2503 raft_consensus.cc:2229] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:12:05.948800 2503 raft_consensus.cc:2258] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:12:05.949807 2503 ts_tablet_manager.cc:1905] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:12:05.951581 2503 ts_tablet_manager.cc:1918] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:12:05.951706 2503 log.cc:1196] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/e93e18717edd481abae1d4b363605d38
I20260809 16:12:05.952078 2503 ts_tablet_manager.cc:1939] T e93e18717edd481abae1d4b363605d38 P a31ba78774054007b38faaa317f9e226: Deleting consensus metadata
I20260809 16:12:05.952569 1717 catalog_manager.cc:4841] TS a31ba78774054007b38faaa317f9e226 (127.16.224.130:37561): tablet e93e18717edd481abae1d4b363605d38 (table test-table [id=e762c34c4f8d4bd7b9c416608cd2ccf5]) successfully deleted
I20260809 16:12:05.954747 2500 tablet_replica.cc:330] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: stopping tablet replica
I20260809 16:12:05.954876 2500 raft_consensus.cc:2229] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:12:05.955013 2500 raft_consensus.cc:2258] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:12:05.956251 2500 ts_tablet_manager.cc:1905] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:12:05.957428 2500 ts_tablet_manager.cc:1918] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:12:05.957502 2500 log.cc:1196] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/e93e18717edd481abae1d4b363605d38
I20260809 16:12:05.957732 2500 ts_tablet_manager.cc:1939] T e93e18717edd481abae1d4b363605d38 P 002f599631b14da9911b14e542d9a9af: Deleting consensus metadata
I20260809 16:12:05.958117 1720 catalog_manager.cc:4841] TS 002f599631b14da9911b14e542d9a9af (127.16.224.131:45009): tablet e93e18717edd481abae1d4b363605d38 (table test-table [id=e762c34c4f8d4bd7b9c416608cd2ccf5]) successfully deleted
W20260809 16:12:05.961163 1717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:46759 (error 108)
I20260809 16:12:05.961457 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 2038
I20260809 16:12:05.961608 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:05.961486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:46759 (local address 127.0.0.1:33912)
0809 16:12:05.961536 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:05.961575 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:46759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:12:05.967383 1717 catalog_manager.cc:4584] TS c8ac0cf30cab4087b0585b3c2e77a2da (127.16.224.129:46759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e93e18717edd481abae1d4b363605d38: Network error: Client connection negotiation failed: client connection to 127.16.224.129:46759: connect: Connection refused (error 111)
I20260809 16:12:05.971458 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 2247
I20260809 16:12:05.981626 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 1696
2026-08-09T16:12:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2098 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:12:06 dist-test-slave-fgb3 krb5kdc[2508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:12:06 dist-test-slave-fgb3 krb5kdc[2508](info): set up 2 sockets
Aug 09 16:12:06 dist-test-slave-fgb3 krb5kdc[2508](info): commencing operation
krb5kdc: starting...
W20260809 16:12:07.117617 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:12:07 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:12:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:12:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/rangeradmin_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/rangeradmin_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/rangerlookup_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/rangerlookup_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
I20260809 16:12:07.274351 17282 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLoTDwK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/postgres -l logfile start
2026-08-09 16:12:09.455 UTC [2557] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:12:09.455 UTC [2557] LOG: listening on IPv4 address "127.16.224.148", port 50523
2026-08-09 16:12:09.458 UTC [2557] LOG: listening on Unix socket "/tmp/.s.PGSQL.50523"
2026-08-09 16:12:09.485 UTC [2561] LOG: database system was shut down at 2026-08-09 16:12:08 UTC
2026-08-09 16:12:09.489 UTC [2557] LOG: database system is ready to accept connections
I20260809 16:12:10.549515 17282 mini_postgres.cc:96] Postgres bound to 50523
2026-08-09 16:12:10.557 UTC [2574] FATAL: database "slave" does not exist
127.16.224.148:50523 - accepting connections
I20260809 16:12:10.558183 17282 mini_ranger.cc:161] Starting Ranger...
I20260809 16:12:10.574875 17282 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:12:10.946586 17282 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:12:10.946671 17282 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-admin
2026-08-09 16:12:11,267 [I] DB FLAVOR :POSTGRES
2026-08-09 16:12:11,267 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:12:11,267 [I] Checking connection..
2026-08-09 16:12:11,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:12:11,545 [I] Checking connection passed.
2026-08-09 16:12:11,545 [I] --------- Verifying version history table ---------
2026-08-09 16:12:11,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:12:11,811 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:12:11,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:12:12,073 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:12:12,073 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:12:12,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:12:12.335 UTC [2661] WARNING: there is no transaction in progress
2026-08-09 16:12:12,341 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:12:12,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:12:12,581 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:12:12,581 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:12:12,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:12:12,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:12:13,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fgb3.c.gcp-upstream.internal','N') ;"
2026-08-09 16:12:13,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:12:13,550 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:12:13,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:12:13,796 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:12:13,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:12:14,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:12:14,288 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:12:14,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:12:15.193 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.208 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.222 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.234 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.248 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.263 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.314 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.321 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.333 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.342 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.351 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.364 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.373 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.379 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.384 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15.390 UTC [2825] WARNING: there is no transaction in progress
2026-08-09 16:12:15,913 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:12:15,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:12:16,165 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:12:16,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:12:16,410 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:12:16,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:12:16,685 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:12:16,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:12:16,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:12:17,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fgb3.c.gcp-upstream.internal';"
2026-08-09 16:12:17,412 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:12:17,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:12:17,648 [I] DB_PATCHES have already been applied
I20260809 16:12:17.654109 17282 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLoTDwK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:12:17.654170 17282 mini_ranger.cc:193] Using host: 127.16.224.148
I20260809 16:12:17.657879 17282 mini_ranger.cc:227] Ranger admin URL: http://127.16.224.148:59137
Aug 09, 2026 4:12:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:12:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:12:18.854770 17282 mini_ranger.cc:160] Time spent starting Ranger: real 8.297s user 0.000s sys 0.010s
Aug 09, 2026 4:12:37 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:12:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 4:12:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:12:37 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:12:38 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:12:39.418678 17282 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/rangerkms_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/rangerkms_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.224.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.224.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:12:39 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291959, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:12:39.497260 17282 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:12:39.507157 17282 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:12:40.194541 17282 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:12:40.194629 17282 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms
I20260809 16:12:40.194645 17282 mini_ranger_kms.cc:227] Using postgres at 127.16.224.148:50523
2026-08-09 16:12:40,431 [I] DB FLAVOR :POSTGRES
2026-08-09 16:12:40,432 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:12:40,432 [I] Checking connection
2026-08-09 16:12:40,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:12:40,673 [I] connection success
2026-08-09 16:12:40,673 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:12:40,673 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:12:40,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:12:40,913 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:12:40,913 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:12:40,913 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:12:40,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:50523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:12:41,193 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:12:41.635771 17282 mini_ranger_kms.cc:310] Created kms service
I20260809 16:12:41.838011 17282 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:12:41.881589 17282 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:12:41 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291959, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:12:42.300019 17282 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:12:42.300143 17282 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:12:42.300161 17282 mini_ranger_kms.cc:240] Using host: 127.16.224.148
I20260809 16:12:42.304453 17282 mini_ranger_kms.cc:276] Ranger KMS PID: 3111
I20260809 16:12:42.304591 17282 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.224.148:52005
Aug 09, 2026 4:12:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:12:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:12:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:12:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:12:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:12:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 16:12:44 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786291964, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.224.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:44.833221 17282 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.336s user 0.008s sys 0.210s
I20260809 16:12:44.833328 17282 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:12:44.833407 17282 mini_ranger_kms.cc:401] 127.16.224.148:52005/kms/v1/keys
Aug 09 16:12:44 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291964, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.224.148@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
Aug 09 16:12:45 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291964, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.224.148@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
Aug 09 16:12:47 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:12:47.538187 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:38961
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:38961
--ranger_config_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:52005/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:47.620855 3188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:47.621071 3188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:47.621099 3188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:47.622229 3188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:12:47.622270 3188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:47.622288 3188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:47.622303 3188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:12:47.622339 3188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:12:47.623993 3188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.224.148:52005/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:38961
--ranger_config_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:38961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:47.624236 3188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:47.624454 3188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:47.629418 3204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:47.629446 3188 server_base.cc:1034] running on GCE node
W20260809 16:12:47.629452 3202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:47.629415 3194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:47.629853 3188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:47.630131 3188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:47.631271 3188 hybrid_clock.cc:648] HybridClock initialized: now 1786291967631257 us; error 30 us; skew 500 ppm
Aug 09 16:12:47 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291967, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:47.643960 3188 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:12:47.644446 3188 webserver.cc:460] Webserver started at http://127.16.224.190:40485/ using document root <none> and password file <none>
I20260809 16:12:47.644649 3188 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:47.644690 3188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:47.644819 3188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:12:47 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291967, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:12:47.698271 3188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "b30a3500fcf34428a7dcb24e4f355ca2"
format_stamp: "Formatted at 2026-08-09 16:12:47 on dist-test-slave-fgb3"
server_key: "f8cfd5acc657a3d4ae2a3781b9a32347"
server_key_iv: "92e85e0f935435f89d54b89952bf07b7"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:47.698760 3188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "b30a3500fcf34428a7dcb24e4f355ca2"
format_stamp: "Formatted at 2026-08-09 16:12:47 on dist-test-slave-fgb3"
server_key: "f8cfd5acc657a3d4ae2a3781b9a32347"
server_key_iv: "92e85e0f935435f89d54b89952bf07b7"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:47.700242 3188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:12:47.715363 3215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:47.715588 3188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:12:47.715770 3188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "b30a3500fcf34428a7dcb24e4f355ca2"
format_stamp: "Formatted at 2026-08-09 16:12:47 on dist-test-slave-fgb3"
server_key: "f8cfd5acc657a3d4ae2a3781b9a32347"
server_key_iv: "92e85e0f935435f89d54b89952bf07b7"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:47.715881 3188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:47.734468 3188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:47.736049 3188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:47.736230 3188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:47.740453 3188 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:38961
I20260809 16:12:47.740487 3286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:38961 every 8 connection(s)
I20260809 16:12:47.740819 3188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:12:47.741490 3288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:47.743795 3288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2: Bootstrap starting.
I20260809 16:12:47.744354 3288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:47.744590 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 3188
I20260809 16:12:47.744630 3288 log.cc:824] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:47.744693 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:12:47.744866 17282 external_mini_cluster.cc:1428] Setting key d2e5ff86ec7d89fe84001dab9389096d
I20260809 16:12:47.745519 3288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2: No bootstrap required, opened a new log
I20260809 16:12:47.747645 3288 raft_consensus.cc:348] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 38961 } }
I20260809 16:12:47.747781 3288 raft_consensus.cc:374] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:47.747814 3288 raft_consensus.cc:729] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b30a3500fcf34428a7dcb24e4f355ca2, State: Initialized, Role: FOLLOWER
I20260809 16:12:47.747939 3288 consensus_queue.cc:260] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 38961 } }
I20260809 16:12:47.748011 3288 raft_consensus.cc:388] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:12:47.748034 3288 raft_consensus.cc:482] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:12:47.748065 3288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:47.748615 3288 raft_consensus.cc:504] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 38961 } }
I20260809 16:12:47.748716 3288 leader_election.cc:302] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b30a3500fcf34428a7dcb24e4f355ca2; no voters:
I20260809 16:12:47.748873 3288 leader_election.cc:288] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:12:47.748915 3295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:12:47.749034 3295 raft_consensus.cc:686] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [term 1 LEADER]: Becoming Leader. State: Replica: b30a3500fcf34428a7dcb24e4f355ca2, State: Running, Role: LEADER
I20260809 16:12:47.749151 3288 sys_catalog.cc:564] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:12:47.749179 3295 consensus_queue.cc:237] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 38961 } }
I20260809 16:12:47.749482 3295 sys_catalog.cc:455] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b30a3500fcf34428a7dcb24e4f355ca2. Latest consensus state: current_term: 1 leader_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 38961 } } }
I20260809 16:12:47.749544 3295 sys_catalog.cc:458] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:47.749742 3297 sys_catalog.cc:455] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30a3500fcf34428a7dcb24e4f355ca2" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 38961 } } }
I20260809 16:12:47.749876 3297 sys_catalog.cc:458] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:12:47.750259 3303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:12:47.750845 3303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:12:47.750865 3288 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 09 16:12:47 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:47.755363 3303 catalog_manager.cc:1347] Generated new cluster ID: 416b8bca37264aaeaa9c45d3fc4edbc8
I20260809 16:12:47.755491 3303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:12:47.758404 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:47.745329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.16.224.190:38961)
0809 16:12:47.745497 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:47.745507 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:47.745536 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:12:47.745971 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:47.745979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:47.746026 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:47.746152 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:47.746865 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:47.747732 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:47.748581 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:47.749160 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:47.749255 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:47.753593 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:47.753673 (+ 80us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:47.753674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:47.753818 (+ 144us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:47.756692 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:47.757008 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:47.757011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:47.757012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:47.757113 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:47.757293 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:47.757294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:47.757294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:47.757618 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:47.757785 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:47.757865 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 16:12:47.758125 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260809 16:12:47.772600 3303 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:12:47.773310 3303 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:12:47.777202 3303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2: Generated new TSK 0
I20260809 16:12:47.777362 3303 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:12:48.472393 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:52005/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:38961
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:48.562640 3355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:48.562827 3355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:48.562848 3355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:48.563930 3355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:48.563977 3355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:48.564000 3355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:12:48.565402 3355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.224.148:52005/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:38961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:48.565599 3355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:48.565807 3355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:48.570672 3366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:48.570672 3369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:48.570704 3372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:48.570941 3355 server_base.cc:1034] running on GCE node
I20260809 16:12:48.571102 3355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:48.571280 3355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:48.572399 3355 hybrid_clock.cc:648] HybridClock initialized: now 1786291968572375 us; error 40 us; skew 500 ppm
Aug 09 16:12:48 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:48.574970 3355 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:12:48.575419 3355 webserver.cc:460] Webserver started at http://127.16.224.129:41721/ using document root <none> and password file <none>
I20260809 16:12:48.575646 3355 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:48.575695 3355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:48.575815 3355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:12:48 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:12:48.596336 3355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "6aaaa388584448a1afe5b5389894c74a"
format_stamp: "Formatted at 2026-08-09 16:12:48 on dist-test-slave-fgb3"
server_key: "07fe66fa0c7ae102338e5535a3074323"
server_key_iv: "02ffe8cf7ae944cf77115bbe230e7454"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:48.596828 3355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "6aaaa388584448a1afe5b5389894c74a"
format_stamp: "Formatted at 2026-08-09 16:12:48 on dist-test-slave-fgb3"
server_key: "07fe66fa0c7ae102338e5535a3074323"
server_key_iv: "02ffe8cf7ae944cf77115bbe230e7454"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:48.598675 3355 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:12:48.616623 3382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:48.616875 3355 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 16:12:48.616983 3355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "6aaaa388584448a1afe5b5389894c74a"
format_stamp: "Formatted at 2026-08-09 16:12:48 on dist-test-slave-fgb3"
server_key: "07fe66fa0c7ae102338e5535a3074323"
server_key_iv: "02ffe8cf7ae944cf77115bbe230e7454"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:48.617082 3355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:48.641813 3355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:48.646247 3355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:48.646498 3355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:48.646795 3355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:48.647221 3355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:48.647264 3355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:48.647295 3355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:48.647333 3355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:48.656248 3355 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:41099
I20260809 16:12:48.656805 3355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:12:48.659986 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 3355
I20260809 16:12:48.660100 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:12:48.660315 17282 external_mini_cluster.cc:1428] Setting key 2dd44cd02650cb2819a47f1f892d6909
I20260809 16:12:48.663992 3549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:41099 every 8 connection(s)
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Aug 09 16:12:48 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:48.679787 3556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:48.664770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:38961 (local address 127.16.224.129:57451)
0809 16:12:48.666445 (+ 1675us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:48.666497 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:48.667289 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:48.667619 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:48.667623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:48.667895 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:48.668214 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:48.668222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.669176 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.669178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:48.670542 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:48.670546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.671075 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.671077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:48.671240 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:48.671865 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:48.671874 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:48.674873 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:48.676878 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:48.676885 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:48.676887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:48.677186 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:48.678568 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:48.678569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:48.678570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:48.678710 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:48.679047 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:48.679056 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:48.679233 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 16:12:48.679346 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:12:48.679518 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":160,"threads_started":1}
I20260809 16:12:48.680539 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:48.664821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:57451 (local address 127.16.224.190:38961)
0809 16:12:48.665000 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:48.665003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:48.666679 (+ 1676us) server_negotiation.cc:407] Connection header received
0809 16:12:48.667425 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:48.667426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:48.667441 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:48.667528 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:48.668331 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.669068 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.670644 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.670967 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.671045 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:48.674984 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:48.674986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:48.674986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:48.674995 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:48.676786 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:48.678410 (+ 1624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:48.678411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:48.678412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:48.678487 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:48.678797 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:48.678798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:48.678799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:48.678952 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:48.679018 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:48.680354 (+ 1336us) server_negotiation.cc:299] Negotiation successful
0809 16:12:48.680446 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260809 16:12:48.681313 3551 heartbeater.cc:347] Connected to a master server at 127.16.224.190:38961
I20260809 16:12:48.681417 3551 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:48.681615 3551 heartbeater.cc:511] Master 127.16.224.190:38961 requested a full tablet report, sending...
I20260809 16:12:48.682233 3241 ts_manager.cc:194] Registered new tserver with Master: 6aaaa388584448a1afe5b5389894c74a (127.16.224.129:41099)
I20260809 16:12:48.683184 3241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:57451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:12:48.690927 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:52005/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:38961
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:48.805115 3559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:48.805410 3559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:48.805452 3559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:48.807192 3559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:48.807261 3559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:48.807305 3559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:12:48.809485 3559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.224.148:52005/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:38961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:48.809780 3559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:48.810091 3559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:48.816823 3571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:48.817081 3572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:48.816834 3575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:48.817977 3559 server_base.cc:1034] running on GCE node
I20260809 16:12:48.818204 3559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:48.818504 3559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:48.819700 3559 hybrid_clock.cc:648] HybridClock initialized: now 1786291968819407 us; error 311 us; skew 500 ppm
Aug 09 16:12:48 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:48.823043 3559 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:12:48.823580 3559 webserver.cc:460] Webserver started at http://127.16.224.130:46191/ using document root <none> and password file <none>
I20260809 16:12:48.823884 3559 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:48.823966 3559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:48.824115 3559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:12:48 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
Aug 09 16:12:48 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:48.838366 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "b0f97bcddd9e4142ae84ad96e42e40f1"
format_stamp: "Formatted at 2026-08-09 16:12:48 on dist-test-slave-fgb3"
server_key: "8f75f08eab2fb73f41267b005a635dd4"
server_key_iv: "09943b7fdb55f193401c3712ba3f2c8f"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:48.838986 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "b0f97bcddd9e4142ae84ad96e42e40f1"
format_stamp: "Formatted at 2026-08-09 16:12:48 on dist-test-slave-fgb3"
server_key: "8f75f08eab2fb73f41267b005a635dd4"
server_key_iv: "09943b7fdb55f193401c3712ba3f2c8f"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:48.844178 3559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 16:12:48.870481 3585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:48.871209 3559 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260809 16:12:48.871345 3559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "b0f97bcddd9e4142ae84ad96e42e40f1"
format_stamp: "Formatted at 2026-08-09 16:12:48 on dist-test-slave-fgb3"
server_key: "8f75f08eab2fb73f41267b005a635dd4"
server_key_iv: "09943b7fdb55f193401c3712ba3f2c8f"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:48.871474 3559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:48.906181 3559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:48.909176 3559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:48.909401 3559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:48.909714 3559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:48.910192 3559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:48.910296 3559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:48.910413 3559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:48.910475 3559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:48.918143 3559 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:42665
I20260809 16:12:48.919131 3559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:12:48.919473 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 3559
I20260809 16:12:48.919581 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:12:48.919788 17282 external_mini_cluster.cc:1428] Setting key a55fdaa481059d156b0c512a704977fe
I20260809 16:12:48.922078 3771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:42665 every 8 connection(s)
Aug 09 16:12:48 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:48.937024 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:48.923734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:49501 (local address 127.16.224.190:38961)
0809 16:12:48.923781 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:48.923783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:48.926636 (+ 2853us) server_negotiation.cc:407] Connection header received
0809 16:12:48.927372 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:48.927374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:48.927391 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:48.927424 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:48.928313 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.929144 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.930743 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.931070 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.931113 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:48.933708 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:48.933710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:48.933710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:48.933717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:48.935346 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:48.935836 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:48.935837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:48.935838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:48.935901 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:48.936226 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:48.936227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:48.936228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:48.936370 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:48.936416 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:48.936830 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 16:12:48.936927 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:12:48.937882 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:48.923452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:38961 (local address 127.16.224.130:49501)
0809 16:12:48.926482 (+ 3030us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:48.926536 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:48.927234 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:48.927636 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:48.927641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:48.927912 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:48.928199 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:48.928205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.929259 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.929260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:48.930642 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:48.930645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:48.931212 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:48.931213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:48.931362 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:48.931893 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:48.931917 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:48.933601 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:48.935438 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:48.935445 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:48.935448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:48.935735 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:48.935972 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:48.935973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:48.935973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:48.936151 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:48.936464 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:48.936471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:48.936640 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 16:12:48.937411 (+ 771us) client_negotiation.cc:240] Negotiation successful
0809 16:12:48.937625 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2978,"thread_start_us":74,"threads_started":1}
I20260809 16:12:48.938638 3772 heartbeater.cc:347] Connected to a master server at 127.16.224.190:38961
I20260809 16:12:48.938740 3772 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:48.938930 3772 heartbeater.cc:511] Master 127.16.224.190:38961 requested a full tablet report, sending...
I20260809 16:12:48.939378 3241 ts_manager.cc:194] Registered new tserver with Master: b0f97bcddd9e4142ae84ad96e42e40f1 (127.16.224.130:42665)
I20260809 16:12:48.939833 3241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:49501
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:12:48.961161 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:52005/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:38961
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:36841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:12:49.085763 3779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:12:49.086130 3779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:12:49.086225 3779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:12:49.088011 3779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:12:49.088243 3779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:12:49.088354 3779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:12:49.090574 3779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:36841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.224.148:52005/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:38961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:12:49.091037 3779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:12:49.091390 3779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:12:49.098371 3788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:49.098721 3786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:12:49.098976 3791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:12:49.099737 3779 server_base.cc:1034] running on GCE node
I20260809 16:12:49.100044 3779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:12:49.100301 3779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:12:49.101457 3779 hybrid_clock.cc:648] HybridClock initialized: now 1786291969101426 us; error 43 us; skew 500 ppm
Aug 09 16:12:49 dist-test-slave-fgb3 krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291969, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:12:49.104853 3779 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:12:49.105353 3779 webserver.cc:460] Webserver started at http://127.16.224.131:33411/ using document root <none> and password file <none>
I20260809 16:12:49.105593 3779 fs_manager.cc:359] Metadata directory not provided
I20260809 16:12:49.105639 3779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:12:49.105756 3779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:12:49 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291969, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:12:49.123344 3779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "17e31d7db56c43abb5108b34a77572c2"
format_stamp: "Formatted at 2026-08-09 16:12:49 on dist-test-slave-fgb3"
server_key: "f5686e286d80d8a629d5db8801ed40d8"
server_key_iv: "537a74f64f38a6904e558070d092e95b"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:49.123971 3779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "17e31d7db56c43abb5108b34a77572c2"
format_stamp: "Formatted at 2026-08-09 16:12:49 on dist-test-slave-fgb3"
server_key: "f5686e286d80d8a629d5db8801ed40d8"
server_key_iv: "537a74f64f38a6904e558070d092e95b"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:49.126677 3779 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:12:49.140760 3801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:49.141007 3779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:12:49.141136 3779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "17e31d7db56c43abb5108b34a77572c2"
format_stamp: "Formatted at 2026-08-09 16:12:49 on dist-test-slave-fgb3"
server_key: "f5686e286d80d8a629d5db8801ed40d8"
server_key_iv: "537a74f64f38a6904e558070d092e95b"
server_key_version: "kuduclusterkey@0"
I20260809 16:12:49.141244 3779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:12:49.188760 3779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:12:49.192139 3779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:12:49.193418 3779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:12:49.193778 3779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:12:49.194342 3779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:12:49.194448 3779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:49.194528 3779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:12:49.194591 3779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:12:49.205363 3779 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:39085
I20260809 16:12:49.205906 3779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:12:49.206585 3977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:39085 every 8 connection(s)
I20260809 16:12:49.210189 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 3779
I20260809 16:12:49.210305 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:12:49.210585 17282 external_mini_cluster.cc:1428] Setting key df42440247aaf28c03fff1a22bc76af2
Aug 09 16:12:49 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291969, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:12:49.219314 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.208293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:40255 (local address 127.16.224.190:38961)
0809 16:12:49.208353 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.208355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.208601 (+ 246us) server_negotiation.cc:407] Connection header received
0809 16:12:49.209480 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.209481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.209494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.209526 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:49.210548 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.211297 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.212972 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.213278 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.213320 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.216001 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:49.216003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:49.216003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:49.216011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:49.217734 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:49.218234 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:49.218235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:49.218236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:49.218294 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:49.218595 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:49.218595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:49.218596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:49.218732 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:49.218771 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.219125 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.219215 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:12:49.220188 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.207986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:38961 (local address 127.16.224.131:40255)
0809 16:12:49.208464 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.208513 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:49.209174 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:49.209719 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:49.209723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:49.210101 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:12:49.210425 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.210434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.211432 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.211433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.212868 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.212871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.213424 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.213425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.213574 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.214158 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:12:49.214169 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:12:49.215888 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:12:49.217830 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:49.217837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:49.217840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:49.218144 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:49.218392 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:12:49.218393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:12:49.218393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:12:49.218524 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:12:49.218813 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:12:49.218818 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:12:49.218979 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 16:12:49.219776 (+ 797us) client_negotiation.cc:240] Negotiation successful
0809 16:12:49.219945 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":90,"threads_started":1}
I20260809 16:12:49.220865 3978 heartbeater.cc:347] Connected to a master server at 127.16.224.190:38961
I20260809 16:12:49.220960 3978 heartbeater.cc:464] Registering TS with master...
I20260809 16:12:49.221146 3978 heartbeater.cc:511] Master 127.16.224.190:38961 requested a full tablet report, sending...
I20260809 16:12:49.221596 3241 ts_manager.cc:194] Registered new tserver with Master: 17e31d7db56c43abb5108b34a77572c2 (127.16.224.131:39085)
I20260809 16:12:49.222106 3241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:40255
I20260809 16:12:49.223565 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:12:49.233156 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.225643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.16.224.190:38961)
0809 16:12:49.225680 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.225681 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.225790 (+ 109us) server_negotiation.cc:407] Connection header received
0809 16:12:49.225877 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.225878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.225887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.225914 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:12:49.226708 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.227441 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.228551 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.228842 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.228883 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.230395 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:12:49.230396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:12:49.230397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:12:49.230404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:12:49.231937 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:49.232343 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:49.232344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:49.232344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:49.232401 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:12:49.232592 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:12:49.232593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:12:49.232593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:12:49.232709 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:12:49.232781 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.232990 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.233063 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:12:49.235273 3241 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:12:49.237627 3241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:12:49.251312 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.242022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:41099 (local address 127.0.0.1:51702)
0809 16:12:49.242790 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.242797 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:49.242829 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:49.246695 (+ 3866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:49.246698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:49.246701 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:49.246794 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.246796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.248161 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.248162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.249890 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.249893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.250990 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.250992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.251083 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.251100 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:12:49.251172 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:12:49.251226 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":100,"threads_started":1}
I20260809 16:12:49.251894 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.242117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51702 (local address 127.16.224.129:41099)
0809 16:12:49.246418 (+ 4301us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.246424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.246437 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:12:49.246477 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.246479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.246496 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.246593 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:49.246912 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.248060 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.249985 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.250811 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.250887 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.250952 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.251765 (+ 813us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.251811 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4259,"thread_start_us":50,"threads_started":1}
I20260809 16:12:49.252840 3455 tablet_service.cc:1467] Processing CreateTablet for tablet 20d9556be59540fbb2dd776093dd0d76 (DEFAULT_TABLE table=test-table [id=ab6bab9a22404387b81036e7653028dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:12:49.253273 3455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d9556be59540fbb2dd776093dd0d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:49.261371 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.241633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:42665 (local address 127.0.0.1:60288)
0809 16:12:49.242689 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.242696 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:49.242729 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:49.257420 (+ 14691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:49.257421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:49.257424 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:49.257472 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.257474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.258939 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.258941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.260071 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.260074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.261127 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.261129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.261193 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.261198 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:12:49.261229 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 16:12:49.261270 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":160,"threads_started":1}
I20260809 16:12:49.261564 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.241402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:39085 (local address 127.0.0.1:38084)
0809 16:12:49.242497 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.242536 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:49.242596 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:49.257828 (+ 15232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:49.257829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:49.257832 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:49.257879 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.257880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.259289 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.259290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.260388 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.260391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.261350 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.261351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.261414 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.261419 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:12:49.261451 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 16:12:49.261490 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"thread_start_us":74,"threads_started":1}
I20260809 16:12:49.261884 4002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.241936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.16.224.131:39085)
0809 16:12:49.257633 (+ 15697us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.257638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.257648 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:12:49.257687 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.257689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.257705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.257952 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:49.258052 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.259185 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.260510 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.261243 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.261661 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.261730 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.261769 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.261806 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15625,"thread_start_us":82,"threads_started":1}
I20260809 16:12:49.262846 3878 tablet_service.cc:1467] Processing CreateTablet for tablet 20d9556be59540fbb2dd776093dd0d76 (DEFAULT_TABLE table=test-table [id=ab6bab9a22404387b81036e7653028dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:12:49.263226 3878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d9556be59540fbb2dd776093dd0d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:49.262271 4008 tablet_bootstrap.cc:492] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a: Bootstrap starting.
I20260809 16:12:49.267308 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.241781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60288 (local address 127.16.224.130:42665)
0809 16:12:49.257241 (+ 15460us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.257246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.257258 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:12:49.257295 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.257297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.257313 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.257395 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:49.257574 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.258829 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.260203 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.261025 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.267082 (+ 6057us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.267154 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.267184 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.267227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15417,"thread_start_us":82,"threads_started":1}
I20260809 16:12:49.267865 4010 tablet_bootstrap.cc:492] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Bootstrap starting.
I20260809 16:12:49.268159 3661 tablet_service.cc:1467] Processing CreateTablet for tablet 20d9556be59540fbb2dd776093dd0d76 (DEFAULT_TABLE table=test-table [id=ab6bab9a22404387b81036e7653028dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:12:49.268530 3661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d9556be59540fbb2dd776093dd0d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:12:49.269294 4010 tablet_bootstrap.cc:654] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:49.269636 4010 log.cc:824] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:49.270483 4010 tablet_bootstrap.cc:492] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: No bootstrap required, opened a new log
I20260809 16:12:49.270545 4010 ts_tablet_manager.cc:1397] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:12:49.272284 4008 tablet_bootstrap.cc:654] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:49.272504 4010 raft_consensus.cc:348] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.272675 4010 raft_consensus.cc:374] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:49.272686 4008 log.cc:824] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:49.272709 4010 raft_consensus.cc:729] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e31d7db56c43abb5108b34a77572c2, State: Initialized, Role: FOLLOWER
I20260809 16:12:49.272827 4010 consensus_queue.cc:260] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.273051 4013 tablet_bootstrap.cc:492] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1: Bootstrap starting.
I20260809 16:12:49.273138 4010 ts_tablet_manager.cc:1428] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:12:49.273470 3978 heartbeater.cc:503] Master 127.16.224.190:38961 was elected leader, sending a full tablet report...
I20260809 16:12:49.274027 4013 tablet_bootstrap.cc:654] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1: Neither blocks nor log segments found. Creating new log.
I20260809 16:12:49.274343 4013 log.cc:824] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:12:49.283186 4008 tablet_bootstrap.cc:492] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a: No bootstrap required, opened a new log
I20260809 16:12:49.283288 4008 ts_tablet_manager.cc:1397] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20260809 16:12:49.285964 4008 raft_consensus.cc:348] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.286154 4008 raft_consensus.cc:374] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:49.286195 4008 raft_consensus.cc:729] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aaaa388584448a1afe5b5389894c74a, State: Initialized, Role: FOLLOWER
I20260809 16:12:49.286479 4008 consensus_queue.cc:260] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.286864 4008 ts_tablet_manager.cc:1428] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 16:12:49.286976 4013 tablet_bootstrap.cc:492] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1: No bootstrap required, opened a new log
I20260809 16:12:49.287061 4013 ts_tablet_manager.cc:1397] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260809 16:12:49.287075 3551 heartbeater.cc:503] Master 127.16.224.190:38961 was elected leader, sending a full tablet report...
I20260809 16:12:49.289390 4013 raft_consensus.cc:348] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.289572 4013 raft_consensus.cc:374] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:12:49.289618 4013 raft_consensus.cc:729] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f97bcddd9e4142ae84ad96e42e40f1, State: Initialized, Role: FOLLOWER
I20260809 16:12:49.289769 4013 consensus_queue.cc:260] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.290189 4013 ts_tablet_manager.cc:1428] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:12:49.290357 3772 heartbeater.cc:503] Master 127.16.224.190:38961 was elected leader, sending a full tablet report...
Aug 09 16:12:49 dist-test-slave-fgb3 krb5kdc[2508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291968, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
W20260809 16:12:49.415141 3552 tablet.cc:2369] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:12:49.423046 3773 tablet.cc:2369] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:12:49.459609 3981 tablet.cc:2369] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:12:49.484448 4022 raft_consensus.cc:482] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:12:49.484615 4022 raft_consensus.cc:504] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.485097 4022 leader_election.cc:288] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e31d7db56c43abb5108b34a77572c2 (127.16.224.131:39085), 6aaaa388584448a1afe5b5389894c74a (127.16.224.129:41099)
I20260809 16:12:49.489282 4002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.485605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:41611 (local address 127.16.224.131:39085)
0809 16:12:49.485651 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.485653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.485776 (+ 123us) server_negotiation.cc:407] Connection header received
0809 16:12:49.485888 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.485889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.485901 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.485929 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:49.486145 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.486924 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.488402 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.488914 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.488959 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.488968 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.489180 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.489215 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:12:49.489615 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.485544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:46429 (local address 127.16.224.129:41099)
0809 16:12:49.485599 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.485601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.485724 (+ 123us) server_negotiation.cc:407] Connection header received
0809 16:12:49.485859 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.485860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.485872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.485905 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:12:49.486135 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.486899 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.488560 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.489189 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.489243 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.489253 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.489493 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.489541 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:12:49.489693 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.485420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:39085 (local address 127.16.224.130:41611)
0809 16:12:49.485710 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.485721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:49.485802 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:49.485995 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:49.485996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:49.485998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:49.486069 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.486072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.487045 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.487046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.488282 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.488285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.489044 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.489045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.489110 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.489114 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:12:49.489568 (+ 454us) client_negotiation.cc:240] Negotiation successful
0809 16:12:49.489612 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":20,"thread_start_us":65,"threads_started":1}
I20260809 16:12:49.490032 3884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d9556be59540fbb2dd776093dd0d76" candidate_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e31d7db56c43abb5108b34a77572c2" is_pre_election: true
I20260809 16:12:49.490134 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.485321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:41099 (local address 127.16.224.130:46429)
0809 16:12:49.485651 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.485663 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:49.485768 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:49.485980 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:12:49.485981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:12:49.485983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:12:49.486057 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.486059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.487022 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.487023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.488301 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:12:49.488304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.489321 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.489322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:12:49.489401 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.489405 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:12:49.489998 (+ 593us) client_negotiation.cc:240] Negotiation successful
0809 16:12:49.490047 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":48,"thread_start_us":77,"threads_started":1}
I20260809 16:12:49.490186 3884 raft_consensus.cc:2454] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f97bcddd9e4142ae84ad96e42e40f1 in term 0.
I20260809 16:12:49.490501 3589 leader_election.cc:302] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e31d7db56c43abb5108b34a77572c2, b0f97bcddd9e4142ae84ad96e42e40f1; no voters:
I20260809 16:12:49.490478 3460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d9556be59540fbb2dd776093dd0d76" candidate_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aaaa388584448a1afe5b5389894c74a" is_pre_election: true
I20260809 16:12:49.490624 4022 raft_consensus.cc:2781] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:12:49.490636 3460 raft_consensus.cc:2454] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f97bcddd9e4142ae84ad96e42e40f1 in term 0.
I20260809 16:12:49.490659 4022 raft_consensus.cc:482] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:12:49.490679 4022 raft_consensus.cc:3037] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:49.491627 4022 raft_consensus.cc:504] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.491770 4022 leader_election.cc:288] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [CANDIDATE]: Term 1 election: Requested vote from peers 17e31d7db56c43abb5108b34a77572c2 (127.16.224.131:39085), 6aaaa388584448a1afe5b5389894c74a (127.16.224.129:41099)
I20260809 16:12:49.491966 3884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d9556be59540fbb2dd776093dd0d76" candidate_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e31d7db56c43abb5108b34a77572c2"
I20260809 16:12:49.491988 3460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d9556be59540fbb2dd776093dd0d76" candidate_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aaaa388584448a1afe5b5389894c74a"
I20260809 16:12:49.492053 3884 raft_consensus.cc:3037] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:49.492074 3460 raft_consensus.cc:3037] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:12:49.492895 3884 raft_consensus.cc:2454] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f97bcddd9e4142ae84ad96e42e40f1 in term 1.
I20260809 16:12:49.493031 3460 raft_consensus.cc:2454] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f97bcddd9e4142ae84ad96e42e40f1 in term 1.
I20260809 16:12:49.493069 3589 leader_election.cc:302] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e31d7db56c43abb5108b34a77572c2, b0f97bcddd9e4142ae84ad96e42e40f1; no voters:
I20260809 16:12:49.493147 4022 raft_consensus.cc:2781] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:12:49.493307 4022 raft_consensus.cc:686] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [term 1 LEADER]: Becoming Leader. State: Replica: b0f97bcddd9e4142ae84ad96e42e40f1, State: Running, Role: LEADER
I20260809 16:12:49.493397 4022 consensus_queue.cc:237] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } }
I20260809 16:12:49.494204 3241 catalog_manager.cc:5495] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f97bcddd9e4142ae84ad96e42e40f1 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0f97bcddd9e4142ae84ad96e42e40f1" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 42665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 } health_report { overall_health: UNKNOWN } } }
I20260809 16:12:49.529675 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.526420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.16.224.130:42665)
0809 16:12:49.526469 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:12:49.526471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:12:49.526483 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:12:49.526555 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:12:49.526556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:12:49.526586 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:12:49.526679 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:12:49.526797 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.527528 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.528681 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:12:49.529001 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:12:49.529083 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:12:49.529256 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:12:49.529405 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:12:49.529512 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:12:49.529533 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 16:12:49.529561 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:12:49.531962 3884 raft_consensus.cc:1261] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 1 FOLLOWER]: Refusing update from remote peer b0f97bcddd9e4142ae84ad96e42e40f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:12:49.531981 3460 raft_consensus.cc:1261] T 20d9556be59540fbb2dd776093dd0d76 P 6aaaa388584448a1afe5b5389894c74a [term 1 FOLLOWER]: Refusing update from remote peer b0f97bcddd9e4142ae84ad96e42e40f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:12:49.532446 4022 consensus_queue.cc:1035] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e31d7db56c43abb5108b34a77572c2" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 39085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:12:49.532577 4028 consensus_queue.cc:1035] T 20d9556be59540fbb2dd776093dd0d76 P b0f97bcddd9e4142ae84ad96e42e40f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6aaaa388584448a1afe5b5389894c74a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 41099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:12:49.536357 4034 mvcc.cc:205] Tried to move back new op lower bound from 7316651907199877120 to 7316651907045457920. Current Snapshot: MvccSnapshot[applied={T|T < 7316651907199877120}]
I20260809 16:12:49.536461 4032 mvcc.cc:205] Tried to move back new op lower bound from 7316651907199877120 to 7316651907045457920. Current Snapshot: MvccSnapshot[applied={T|T < 7316651907199877120}]
I20260809 16:12:49.539214 3241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:12:49.539311 3241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:12:49.540149 3241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b30a3500fcf34428a7dcb24e4f355ca2: Sending DeleteTablet for 3 replicas of tablet 20d9556be59540fbb2dd776093dd0d76
I20260809 16:12:49.540535 3661 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d9556be59540fbb2dd776093dd0d76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:12:49 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:60288
I20260809 16:12:49.540530 3878 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d9556be59540fbb2dd776093dd0d76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:12:49 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:38084
I20260809 16:12:49.540933 3455 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d9556be59540fbb2dd776093dd0d76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:12:49 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:51702
I20260809 16:12:49.541611 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 3355
I20260809 16:12:49.545413 4048 tablet_replica.cc:330] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: stopping tablet replica
I20260809 16:12:49.547582 4048 raft_consensus.cc:2229] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:12:49.547739 4048 raft_consensus.cc:2258] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:12:49.550100 4048 ts_tablet_manager.cc:1905] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:12:49.555721 4048 ts_tablet_manager.cc:1918] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:12:49.555895 4048 log.cc:1196] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/20d9556be59540fbb2dd776093dd0d76
W20260809 16:12:49.556303 3592 connection.cc:535] client connection to 127.16.224.129:41099 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:12:49.556409 3592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:12:49.556530 4048 ts_tablet_manager.cc:1939] T 20d9556be59540fbb2dd776093dd0d76 P 17e31d7db56c43abb5108b34a77572c2: Deleting consensus metadata
W20260809 16:12:49.556926 3218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:41099 (error 108)
I20260809 16:12:49.557080 3217 catalog_manager.cc:4841] TS 17e31d7db56c43abb5108b34a77572c2 (127.16.224.131:39085): tablet 20d9556be59540fbb2dd776093dd0d76 (table test-table [id=ab6bab9a22404387b81036e7653028dd]) successfully deleted
I20260809 16:12:49.557574 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 3559
I20260809 16:12:49.560837 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:12:49.560703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:41099 (local address 127.0.0.1:51712)
0809 16:12:49.560754 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.560799 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:41099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 16:12:49.561017 3218 catalog_manager.cc:4584] TS 6aaaa388584448a1afe5b5389894c74a (127.16.224.129:41099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20d9556be59540fbb2dd776093dd0d76: Network error: Client connection negotiation failed: client connection to 127.16.224.129:41099: connect: Connection refused (error 111)
W20260809 16:12:49.570286 3995 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:12:49.570000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:42665 (local address 127.0.0.1:60302)
0809 16:12:49.570054 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:12:49.570065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:12:49.570106 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:12:49.570223 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.130:42665: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:12:49.570889 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 3779
I20260809 16:12:49.579108 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 3188
I20260809 16:12:49.594379 17282 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:12:49.939043 17282 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:12:49.939142 17282 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:12:50.282936 17282 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:12:50.283 UTC [2557] LOG: received smart shutdown request
2026-08-09 16:12:50.286 UTC [2557] LOG: background worker "logical replication launcher" (PID 2570) exited with exit code 1
2026-08-09 16:12:50.287 UTC [2563] LOG: shutting down
2026-08-09 16:12:50.407 UTC [2557] LOG: database system is shut down
2026-08-09T16:12:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44514 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:12:50 dist-test-slave-fgb3 krb5kdc[4062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:12:50 dist-test-slave-fgb3 krb5kdc[4062](info): set up 2 sockets
Aug 09 16:12:50 dist-test-slave-fgb3 krb5kdc[4062](info): commencing operation
krb5kdc: starting...
W20260809 16:12:51.639779 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:12:51 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786291971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:12:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/rangeradmin_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/rangeradmin_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/rangerlookup_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/rangerlookup_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
I20260809 16:12:51.793931 17282 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLoTDwK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/postgres -l logfile start
2026-08-09 16:12:54.014 UTC [4111] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:12:54.014 UTC [4111] LOG: listening on IPv4 address "127.16.224.148", port 45483
2026-08-09 16:12:54.017 UTC [4111] LOG: listening on Unix socket "/tmp/.s.PGSQL.45483"
2026-08-09 16:12:54.038 UTC [4118] LOG: database system was shut down at 2026-08-09 16:12:52 UTC
2026-08-09 16:12:54.041 UTC [4111] LOG: database system is ready to accept connections
I20260809 16:12:55.095742 17282 mini_postgres.cc:96] Postgres bound to 45483
2026-08-09 16:12:55.102 UTC [4128] FATAL: database "slave" does not exist
127.16.224.148:45483 - accepting connections
I20260809 16:12:55.103045 17282 mini_ranger.cc:161] Starting Ranger...
I20260809 16:12:55.117640 17282 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:12:55.510200 17282 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:12:55.510283 17282 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-admin
2026-08-09 16:12:55,776 [I] DB FLAVOR :POSTGRES
2026-08-09 16:12:55,776 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:12:55,776 [I] Checking connection..
2026-08-09 16:12:55,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:12:56,020 [I] Checking connection passed.
2026-08-09 16:12:56,020 [I] --------- Verifying version history table ---------
2026-08-09 16:12:56,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:12:56,280 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:12:56,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:12:56,530 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:12:56,531 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:12:56,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:12:56.774 UTC [4227] WARNING: there is no transaction in progress
2026-08-09 16:12:56,782 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:12:56,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:12:57,013 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:12:57,013 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:12:57,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:12:57,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:12:57,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fgb3.c.gcp-upstream.internal','N') ;"
2026-08-09 16:12:57,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:12:58,001 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:12:58,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:12:58,248 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:12:58,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:12:58,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:12:58,769 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:12:58,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:12:59.646 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.660 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.674 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.686 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.700 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.716 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.764 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.770 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.782 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.791 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.800 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.813 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.822 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.828 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.833 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:12:59.839 UTC [4386] WARNING: there is no transaction in progress
2026-08-09 16:13:00,422 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:13:00,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:13:00,676 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:13:00,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:13:00,939 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:13:00,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:13:01,190 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:13:01,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:13:01,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:13:01,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fgb3.c.gcp-upstream.internal';"
2026-08-09 16:13:01,943 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:13:01,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:13:02,175 [I] DB_PATCHES have already been applied
I20260809 16:13:02.180585 17282 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLoTDwK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:13:02.180655 17282 mini_ranger.cc:193] Using host: 127.16.224.148
I20260809 16:13:02.184662 17282 mini_ranger.cc:227] Ranger admin URL: http://127.16.224.148:47849
Aug 09, 2026 4:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:13:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:13:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:13:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:13:03.450073 17282 mini_ranger.cc:160] Time spent starting Ranger: real 8.347s user 0.000s sys 0.010s
Aug 09, 2026 4:13:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:13:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 4:13:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:13:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:13:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:13:24.948643 17282 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.224.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.224.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/rangerkms_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.224.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/rangerkms_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.224.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.224.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.224.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/HTTP_127.16.224.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:13:25 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292005, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:13:25.037104 17282 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:13:25.046541 17282 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:13:25.716787 17282 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:13:25.716857 17282 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms
I20260809 16:13:25.716869 17282 mini_ranger_kms.cc:227] Using postgres at 127.16.224.148:45483
2026-08-09 16:13:25,876 [I] DB FLAVOR :POSTGRES
2026-08-09 16:13:25,876 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:13:25,877 [I] Checking connection
2026-08-09 16:13:25,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:13:26,113 [I] connection success
2026-08-09 16:13:26,113 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:13:26,113 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:13:26,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:13:26,387 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:13:26,387 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:13:26,388 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:13:26,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.224.148:45483/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:13:26,805 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:13:27.225445 17282 mini_ranger_kms.cc:310] Created kms service
I20260809 16:13:27.379375 17282 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:13:27.418509 17282 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:13:27 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292005, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:27.778882 17282 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:13:27.779098 17282 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLoTDwK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLoTDwK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:13:27.779121 17282 mini_ranger_kms.cc:240] Using host: 127.16.224.148
I20260809 16:13:27.783002 17282 mini_ranger_kms.cc:276] Ranger KMS PID: 4675
I20260809 16:13:27.783074 17282 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.224.148:41643
Aug 09, 2026 4:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:13:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:13:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 16:13:29 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.224.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:30.370189 17282 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.333s user 0.005s sys 0.137s
I20260809 16:13:30.370299 17282 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:13:30.370371 17282 mini_ranger_kms.cc:401] 127.16.224.148:41643/kms/v1/keys
Aug 09 16:13:30 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.224.148@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
Aug 09 16:13:30 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.224.148@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
Aug 09 16:13:32 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:13:33.022979 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44171
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:44171
--ranger_config_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:33.108096 4751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:33.108297 4751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:33.108318 4751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:33.109449 4751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:33.109491 4751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:33.109504 4751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:33.109514 4751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:33.109525 4751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:33.111131 4751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.224.148:41643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:44171
--ranger_config_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:33.111370 4751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:33.111580 4751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:33.116662 4763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:33.116676 4751 server_base.cc:1034] running on GCE node
W20260809 16:13:33.116770 4760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:33.116793 4759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:33.117210 4751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:33.117539 4751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:33.118676 4751 hybrid_clock.cc:648] HybridClock initialized: now 1786292013118650 us; error 43 us; skew 500 ppm
Aug 09 16:13:33 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292013, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:33.121230 4751 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:13:33.121765 4751 webserver.cc:460] Webserver started at http://127.16.224.190:38657/ using document root <none> and password file <none>
I20260809 16:13:33.122008 4751 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:33.122061 4751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:33.122195 4751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:13:33 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292013, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:33.175796 4751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "58bb589e4a194f25a1ac08a8bc629c8a"
format_stamp: "Formatted at 2026-08-09 16:13:33 on dist-test-slave-fgb3"
server_key: "f3e61a311affeece0f119f1885363a4e"
server_key_iv: "205e9e13621953c47a426e1e692cd8d8"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:33.176311 4751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "58bb589e4a194f25a1ac08a8bc629c8a"
format_stamp: "Formatted at 2026-08-09 16:13:33 on dist-test-slave-fgb3"
server_key: "f3e61a311affeece0f119f1885363a4e"
server_key_iv: "205e9e13621953c47a426e1e692cd8d8"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:33.178174 4751 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 16:13:33.193428 4771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:33.193663 4751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:33.193771 4751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "58bb589e4a194f25a1ac08a8bc629c8a"
format_stamp: "Formatted at 2026-08-09 16:13:33 on dist-test-slave-fgb3"
server_key: "f3e61a311affeece0f119f1885363a4e"
server_key_iv: "205e9e13621953c47a426e1e692cd8d8"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:33.193879 4751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:33.212106 4751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:33.214156 4751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:33.214357 4751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:33.219393 4751 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:44171
I20260809 16:13:33.219429 4845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:44171 every 8 connection(s)
I20260809 16:13:33.219846 4751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:13:33.220607 4846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:33.223022 4846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Bootstrap starting.
I20260809 16:13:33.223619 4846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:33.223888 4846 log.cc:824] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:33.224572 4846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: No bootstrap required, opened a new log
I20260809 16:13:33.226629 4846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:33.226780 4846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:33.226802 4846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58bb589e4a194f25a1ac08a8bc629c8a, State: Initialized, Role: FOLLOWER
I20260809 16:13:33.226934 4846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:33.227005 4846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:13:33.227028 4846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:13:33.227073 4846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:33.227602 4846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:33.227692 4846 leader_election.cc:302] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58bb589e4a194f25a1ac08a8bc629c8a; no voters:
I20260809 16:13:33.227852 4846 leader_election.cc:288] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:13:33.227890 4851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:13:33.228044 4851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 LEADER]: Becoming Leader. State: Replica: 58bb589e4a194f25a1ac08a8bc629c8a, State: Running, Role: LEADER
I20260809 16:13:33.228114 4846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:33.228185 4851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:33.228572 4851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58bb589e4a194f25a1ac08a8bc629c8a. Latest consensus state: current_term: 1 leader_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } } }
I20260809 16:13:33.228631 4854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } } }
I20260809 16:13:33.228736 4851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:33.228747 4854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:33.228977 4867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:13:33.229319 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 4751
I20260809 16:13:33.229409 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:13:33.229579 17282 external_mini_cluster.cc:1428] Setting key d9cc301b30d5c4e4253bb532af1c1064
I20260809 16:13:33.230787 4867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:13:33.231086 4846 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:13:33.232589 4867 catalog_manager.cc:1347] Generated new cluster ID: ccd492516dab4687a3378a1ec7239c3d
I20260809 16:13:33.232645 4867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:13:33.247828 4867 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:13:33.249037 4867 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 16:13:33 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:33.256829 4867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Generated new TSK 0
I20260809 16:13:33.257076 4867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:13:33.261849 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:33.230052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.16.224.190:44171)
0809 16:13:33.230355 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:33.230366 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:33.231937 (+ 1571us) server_negotiation.cc:407] Connection header received
0809 16:13:33.232294 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:33.232302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:33.232362 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:33.232472 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:33.243477 (+ 11005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:33.245446 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:33.247185 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:33.247856 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:33.247944 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:33.257238 (+ 9294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:33.257245 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:33.257245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:33.257259 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:33.259904 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:33.260291 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:33.260294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:33.260295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:33.260406 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:33.260588 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:33.260589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:33.260589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:33.260864 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:33.260986 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:33.261468 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 16:13:33.261618 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:13:34.027289 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44171
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:34.129978 4923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:34.130172 4923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:34.130203 4923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:34.131379 4923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:34.131426 4923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:34.131460 4923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:13:34.132881 4923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.224.148:41643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:34.133101 4923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:34.133308 4923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:34.139168 4932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:34.139173 4929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:34.139587 4930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:34.140245 4923 server_base.cc:1034] running on GCE node
I20260809 16:13:34.140440 4923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:34.140671 4923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:34.141832 4923 hybrid_clock.cc:648] HybridClock initialized: now 1786292014141819 us; error 33 us; skew 500 ppm
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:34.144348 4923 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:13:34.144778 4923 webserver.cc:460] Webserver started at http://127.16.224.129:45479/ using document root <none> and password file <none>
I20260809 16:13:34.145035 4923 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:34.145155 4923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:34.145344 4923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:34.165244 4923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "0a6afa2c368b48d7a47a0f7682618fc1"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "564055e421152cfda7fafc093e720e3b"
server_key_iv: "75ad7e0fcaf4b3ec0e339083eed25232"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.165841 4923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "0a6afa2c368b48d7a47a0f7682618fc1"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "564055e421152cfda7fafc093e720e3b"
server_key_iv: "75ad7e0fcaf4b3ec0e339083eed25232"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.173183 4923 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.004s
I20260809 16:13:34.186471 4948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.186719 4923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:13:34.186820 4923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "0a6afa2c368b48d7a47a0f7682618fc1"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "564055e421152cfda7fafc093e720e3b"
server_key_iv: "75ad7e0fcaf4b3ec0e339083eed25232"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.186926 4923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:34.250296 4923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:34.252960 4923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:34.253201 4923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:34.253513 4923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:34.253980 4923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:34.254079 4923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.254181 4923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:34.254256 4923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.261063 4923 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:33309
I20260809 16:13:34.261585 4923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:13:34.263111 5121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:33309 every 8 connection(s)
I20260809 16:13:34.265690 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 4923
I20260809 16:13:34.265810 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:13:34.265997 17282 external_mini_cluster.cc:1428] Setting key 7c6a7fce0b3f06d78dd0d62314582411
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:34.294540 5125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:34.263725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44171 (local address 127.16.224.129:35757)
0809 16:13:34.264306 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:34.264382 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:34.265066 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:34.278720 (+ 13654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:34.278726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:34.278998 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:34.279312 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:34.279320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.284447 (+ 5127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.284449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:34.285852 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:34.285856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.286419 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.286421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:34.286583 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:34.287216 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:34.287227 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:34.290796 (+ 3569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:34.292908 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:34.292916 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:34.292919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:34.293209 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:34.293418 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:34.293419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:34.293420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:34.293549 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:34.293823 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:34.293829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:34.294010 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 16:13:34.294104 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 16:13:34.294264 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":90,"threads_started":1}
I20260809 16:13:34.295321 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:34.263742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:35757 (local address 127.16.224.190:44171)
0809 16:13:34.265286 (+ 1544us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:34.265288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:34.265298 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:13:34.265326 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:34.265327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:34.278473 (+ 13146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:34.278572 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:34.283436 (+ 4864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.284285 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.285968 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.286287 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.286375 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:34.290940 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:34.290943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:34.290943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:34.290954 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:34.292800 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:34.293290 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:34.293291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:34.293291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:34.293353 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:34.293613 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:34.293614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:34.293614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:34.293756 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:34.295089 (+ 1333us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:34.295134 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 16:13:34.295222 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1515,"thread_start_us":66,"threads_started":1}
I20260809 16:13:34.295897 5122 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44171
I20260809 16:13:34.295996 5122 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:34.296193 5122 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:34.296764 4792 ts_manager.cc:194] Registered new tserver with Master: 0a6afa2c368b48d7a47a0f7682618fc1 (127.16.224.129:33309)
I20260809 16:13:34.297663 4792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:35757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:13:34.319662 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44171
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 16:13:34.435812 5129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:34.436080 5129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:34.436115 5129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:34.437758 5129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:34.437815 5129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:34.437852 5129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:13:34.440003 5129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.224.148:41643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:34.440392 5129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:34.440659 5129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:34.447649 5137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:34.447651 5144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:34.448109 5139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:34.448644 5129 server_base.cc:1034] running on GCE node
I20260809 16:13:34.448882 5129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:34.449174 5129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:34.450361 5129 hybrid_clock.cc:648] HybridClock initialized: now 1786292014450322 us; error 50 us; skew 500 ppm
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:34.453644 5129 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:13:34.454169 5129 webserver.cc:460] Webserver started at http://127.16.224.130:37059/ using document root <none> and password file <none>
I20260809 16:13:34.454493 5129 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:34.454602 5129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:34.454743 5129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:34.472771 5129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "34b5d4ff7f92410493b27ced59601028"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "d3725f1548641c4783bcf86719bcdbee"
server_key_iv: "e91a8f459412fae8e42085f73e6fd533"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.473276 5129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "34b5d4ff7f92410493b27ced59601028"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "d3725f1548641c4783bcf86719bcdbee"
server_key_iv: "e91a8f459412fae8e42085f73e6fd533"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.486833 5129 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.000s sys 0.004s
I20260809 16:13:34.502486 5153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.502806 5129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:13:34.503005 5129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "34b5d4ff7f92410493b27ced59601028"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "d3725f1548641c4783bcf86719bcdbee"
server_key_iv: "e91a8f459412fae8e42085f73e6fd533"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.503186 5129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:34.525369 5129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:34.527925 5129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:34.528216 5129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:34.528549 5129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:34.529050 5129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:34.529192 5129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.529273 5129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:34.529330 5129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.536598 5129 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:39827
I20260809 16:13:34.537117 5129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:13:34.538663 5334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:39827 every 8 connection(s)
I20260809 16:13:34.538753 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 5129
I20260809 16:13:34.538857 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:13:34.539063 17282 external_mini_cluster.cc:1428] Setting key f958753f624e366da996d24d3396f1c4
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:34.550390 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:34.538434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:37479 (local address 127.16.224.190:44171)
0809 16:13:34.538496 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:34.538499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:34.539141 (+ 642us) server_negotiation.cc:407] Connection header received
0809 16:13:34.539747 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:34.539748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:34.539762 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:34.539832 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:34.540599 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.541353 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.542826 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.543073 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.543134 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:34.547238 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:34.547240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:34.547240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:34.547248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:34.548836 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:34.549230 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:34.549231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:34.549231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:34.549287 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:34.549585 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:34.549586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:34.549586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:34.549729 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:34.549784 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:34.550163 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 16:13:34.550254 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:13:34.550523 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:34.538503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44171 (local address 127.16.224.130:37479)
0809 16:13:34.539005 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:34.539052 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:34.539651 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:34.539959 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:34.539964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:34.540212 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:34.540503 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:34.540511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.541495 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.541497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:34.542738 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:34.542741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.543218 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.543219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:34.543361 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:34.545414 (+ 2053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:34.545424 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:34.547133 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:34.548922 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:34.548929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:34.548931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:34.549149 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:34.549344 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:34.549345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:34.549346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:34.549475 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:34.549821 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:34.549828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:34.550002 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 16:13:34.550112 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:13:34.550261 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":102,"threads_started":1}
I20260809 16:13:34.550935 5337 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44171
I20260809 16:13:34.551041 5337 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:34.551242 5337 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
I20260809 16:13:34.551632 4792 ts_manager.cc:194] Registered new tserver with Master: 34b5d4ff7f92410493b27ced59601028 (127.16.224.130:39827)
I20260809 16:13:34.552112 4792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:37479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:13:34.572762 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44171
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:34.682021 5343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:34.682329 5343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:34.682375 5343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:34.684128 5343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:34.684187 5343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:34.684224 5343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:13:34.686475 5343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.224.148:41643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:34.686708 5343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:34.686962 5343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:34.693408 5349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:34.693676 5343 server_base.cc:1034] running on GCE node
W20260809 16:13:34.693457 5350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:34.693913 5352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:34.694134 5343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:34.694444 5343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:34.695616 5343 hybrid_clock.cc:648] HybridClock initialized: now 1786292014695593 us; error 37 us; skew 500 ppm
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:34.698900 5343 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:13:34.699433 5343 webserver.cc:460] Webserver started at http://127.16.224.131:43361/ using document root <none> and password file <none>
I20260809 16:13:34.699692 5343 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:34.699754 5343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:34.699898 5343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:34.714928 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "baba9f7916a841299d65965e548d01ee"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "6556fb7040bb5a4b981d40cad40937f6"
server_key_iv: "4c5d384f8078a2444ad7d328d3f93d8e"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.715441 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "baba9f7916a841299d65965e548d01ee"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "6556fb7040bb5a4b981d40cad40937f6"
server_key_iv: "4c5d384f8078a2444ad7d328d3f93d8e"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.717605 5343 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:34.734457 5363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.734637 5343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:13:34.734745 5343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "baba9f7916a841299d65965e548d01ee"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "6556fb7040bb5a4b981d40cad40937f6"
server_key_iv: "4c5d384f8078a2444ad7d328d3f93d8e"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.734854 5343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:34.757649 5343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:34.760027 5343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:34.760169 5343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:34.760401 5343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:34.760795 5343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:34.760848 5343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.760885 5343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:34.760912 5343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.768539 5343 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:38219
I20260809 16:13:34.769006 5343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:13:34.770030 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 5343
I20260809 16:13:34.770144 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:13:34.770401 17282 external_mini_cluster.cc:1428] Setting key 4f7cd15a6a917061b2376ae0fe231ddc
I20260809 16:13:34.780746 5570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:38219 every 8 connection(s)
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:34.792241 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:34.781671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:43615 (local address 127.16.224.190:44171)
0809 16:13:34.781735 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:34.781737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:34.782442 (+ 705us) server_negotiation.cc:407] Connection header received
0809 16:13:34.783143 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:34.783145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:34.783158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:34.783188 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:34.784072 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.784821 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.786356 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.786549 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.786608 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:34.789187 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:34.789189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:34.789189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:34.789196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:34.790767 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:34.791215 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:34.791217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:34.791217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:34.791271 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:34.791522 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:34.791524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:34.791524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:34.791676 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:34.791755 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:34.792060 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 16:13:34.792144 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:13:34.792601 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:34.781571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44171 (local address 127.16.224.131:43615)
0809 16:13:34.782280 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:34.782354 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:34.783023 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:34.783399 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:34.783403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:34.783673 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:34.783959 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:34.783966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.784942 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.784943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:34.786225 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:34.786228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:34.786640 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:34.786641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:34.786774 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:34.787400 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:34.787410 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:34.789084 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:34.790861 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:34.790867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:34.790869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:34.791135 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:34.791335 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:34.791335 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:34.791335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:34.791449 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:34.791747 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:34.791752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:34.791910 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 16:13:34.792020 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:13:34.792164 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":106,"threads_started":1}
I20260809 16:13:34.793196 5572 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44171
I20260809 16:13:34.793292 5572 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:34.793473 5572 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:34.793887 4792 ts_manager.cc:194] Registered new tserver with Master: baba9f7916a841299d65965e548d01ee (127.16.224.131:38219)
I20260809 16:13:34.794413 4792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:43615
I20260809 16:13:34.803928 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:13:34.804576 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 4923
I20260809 16:13:34.814117 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 5129
I20260809 16:13:34.825996 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 5343
I20260809 16:13:34.840238 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 4751
I20260809 16:13:34.856779 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44171
--webserver_interface=127.16.224.190
--webserver_port=38657
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:44171
--ranger_config_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:34.938405 5578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:34.938613 5578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:34.938637 5578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:34.939762 5578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:34.939803 5578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:34.939816 5578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:34.939838 5578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:34.939859 5578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:34.941447 5578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:44171
--ranger_config_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=38657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:34.941669 5578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:34.941869 5578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:34.946560 5587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:34.946569 5578 server_base.cc:1034] running on GCE node
W20260809 16:13:34.946539 5595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:34.946523 5599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:34.946985 5578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:34.947237 5578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:34.948365 5578 hybrid_clock.cc:648] HybridClock initialized: now 1786292014948357 us; error 28 us; skew 500 ppm
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:34.950805 5578 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:13:34.951225 5578 webserver.cc:460] Webserver started at http://127.16.224.190:38657/ using document root <none> and password file <none>
I20260809 16:13:34.951443 5578 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:34.951511 5578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:13:34 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292014, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:34.986812 5578 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:13:34.987713 5608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:34.987923 5578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:13:34.988026 5578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "58bb589e4a194f25a1ac08a8bc629c8a"
format_stamp: "Formatted at 2026-08-09 16:13:33 on dist-test-slave-fgb3"
server_key: "f3e61a311affeece0f119f1885363a4e"
server_key_iv: "205e9e13621953c47a426e1e692cd8d8"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:34.988487 5578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:35.028123 5578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:35.030272 5578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:35.030527 5578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:35.035673 5578 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:44171
I20260809 16:13:35.035732 5697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:44171 every 8 connection(s)
I20260809 16:13:35.036334 5578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:13:35.037650 5700 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:13:35.038352 5700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Bootstrap starting.
I20260809 16:13:35.043076 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 5578
I20260809 16:13:35.043426 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:33309
--local_ip_for_outbound_sockets=127.16.224.129
--tserver_master_addrs=127.16.224.190:44171
--webserver_port=45479
--webserver_interface=127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:35.046717 5700 log.cc:824] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:35.048441 5700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:13:35.048720 5700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Bootstrap complete.
I20260809 16:13:35.051069 5700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:35.051312 5700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58bb589e4a194f25a1ac08a8bc629c8a, State: Initialized, Role: FOLLOWER
I20260809 16:13:35.051524 5700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:35.051605 5700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:13:35.051672 5700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:13:35.051753 5700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:13:35.052841 5700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:35.052947 5700 leader_election.cc:302] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58bb589e4a194f25a1ac08a8bc629c8a; no voters:
I20260809 16:13:35.053164 5700 leader_election.cc:288] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:13:35.053215 5707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:13:35.053403 5707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [term 2 LEADER]: Becoming Leader. State: Replica: 58bb589e4a194f25a1ac08a8bc629c8a, State: Running, Role: LEADER
I20260809 16:13:35.053561 5700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:35.053555 5707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } }
I20260809 16:13:35.053881 5707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58bb589e4a194f25a1ac08a8bc629c8a. Latest consensus state: current_term: 2 leader_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } } }
I20260809 16:13:35.053941 5707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:35.054232 5708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bb589e4a194f25a1ac08a8bc629c8a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44171 } } }
I20260809 16:13:35.054376 5708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:35.054528 5712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:13:35.055078 5712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:13:35.055265 5712 catalog_manager.cc:1259] Loaded cluster ID: ccd492516dab4687a3378a1ec7239c3d
I20260809 16:13:35.055337 5712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:13:35.056521 5712 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:13:35.056751 5712 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Loaded TSK: 0
I20260809 16:13:35.057034 5712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:13:35.058740 5700 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 16:13:35.165355 5704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:35.165730 5704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:35.165817 5704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:35.167621 5704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:35.167737 5704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:35.167829 5704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:13:35.169996 5704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:33309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=45479
--tserver_master_addrs=127.16.224.190:44171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:35.170545 5704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:35.170907 5704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:35.177734 5770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:35.178130 5774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:35.178654 5771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:35.178869 5704 server_base.cc:1034] running on GCE node
I20260809 16:13:35.179241 5704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:35.179518 5704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:35.180663 5704 hybrid_clock.cc:648] HybridClock initialized: now 1786292015180637 us; error 39 us; skew 500 ppm
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:35.183889 5704 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:13:35.184367 5704 webserver.cc:460] Webserver started at http://127.16.224.129:45479/ using document root <none> and password file <none>
I20260809 16:13:35.184599 5704 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:35.184659 5704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:35.208230 5704 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260809 16:13:35.209972 5787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.210242 5704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:13:35.210377 5704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "0a6afa2c368b48d7a47a0f7682618fc1"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "564055e421152cfda7fafc093e720e3b"
server_key_iv: "75ad7e0fcaf4b3ec0e339083eed25232"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:35.210909 5704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:35.237356 5704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:35.239753 5704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:35.239970 5704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:35.240199 5704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:35.240577 5704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:35.240649 5704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.240705 5704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:35.240747 5704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.246995 5704 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:33309
I20260809 16:13:35.247498 5704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:13:35.255359 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 5704
I20260809 16:13:35.255677 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:39827
--local_ip_for_outbound_sockets=127.16.224.130
--tserver_master_addrs=127.16.224.190:44171
--webserver_port=37059
--webserver_interface=127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:35.259187 5954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:33309 every 8 connection(s)
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:35.284698 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:35.260018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44171 (local address 127.16.224.129:34783)
0809 16:13:35.260202 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:35.260245 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:35.260719 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:35.271763 (+ 11044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:35.271768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:35.272007 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:35.272626 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:35.272635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.274299 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.274300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:35.275626 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:35.275629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.276633 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.276635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:35.276767 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:35.277899 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:35.277910 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:35.279562 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:35.282037 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:35.282044 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:35.282047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:35.282443 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:35.282736 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:35.282737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:35.282737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:35.282863 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:35.283369 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:35.283375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:35.284168 (+ 793us) client_negotiation.cc:769] Sending connection context
0809 16:13:35.284277 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 16:13:35.284446 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260809 16:13:35.286065 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:35.260072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:34783 (local address 127.16.224.190:44171)
0809 16:13:35.270865 (+ 10793us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:35.270877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:35.270928 (+ 51us) server_negotiation.cc:407] Connection header received
0809 16:13:35.271093 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:35.271101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:35.271164 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:35.271557 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:35.272775 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.274177 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.275722 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.276395 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.276513 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:35.279688 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:35.279692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:35.279693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:35.279705 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:35.281945 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.282548 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.282552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.282552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.282662 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.282935 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.282936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.282937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.283254 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:35.283339 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:35.285311 (+ 1972us) server_negotiation.cc:299] Negotiation successful
0809 16:13:35.285643 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10520,"thread_start_us":94,"threads_started":1}
I20260809 16:13:35.286913 5955 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44171
I20260809 16:13:35.287009 5955 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.287185 5955 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
W20260809 16:13:35.288085 5955 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:44171 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:13:35.288153 5955 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.288211 5955 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:35.288601 5955 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.288677 5955 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
W20260809 16:13:35.288990 5955 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:13:35.379833 5957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:35.380198 5957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:35.380304 5957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:35.382052 5957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:35.382169 5957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:35.382261 5957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:13:35.384491 5957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:39827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=37059
--tserver_master_addrs=127.16.224.190:44171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:35.384943 5957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:35.385267 5957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:35.394276 5974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:35.394217 5967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:35.394651 5969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:35.395159 5957 server_base.cc:1034] running on GCE node
I20260809 16:13:35.395358 5957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:35.395603 5957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:35.396752 5957 hybrid_clock.cc:648] HybridClock initialized: now 1786292015396727 us; error 40 us; skew 500 ppm
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:35.399643 5957 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:13:35.400100 5957 webserver.cc:460] Webserver started at http://127.16.224.130:37059/ using document root <none> and password file <none>
I20260809 16:13:35.400352 5957 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:35.400419 5957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:35.417713 5957 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 16:13:35.418656 5983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.418802 5957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:13:35.418908 5957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "34b5d4ff7f92410493b27ced59601028"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "d3725f1548641c4783bcf86719bcdbee"
server_key_iv: "e91a8f459412fae8e42085f73e6fd533"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:35.419387 5957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:35.437752 5957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:35.440282 5957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:35.440511 5957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:35.440783 5957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:35.441239 5957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:35.441345 5957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.441422 5957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:35.441500 5957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.449182 5957 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:39827
I20260809 16:13:35.449790 5957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:13:35.452255 6144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:39827 every 8 connection(s)
I20260809 16:13:35.457216 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 5957
I20260809 16:13:35.457767 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.224.148:41643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:38219
--local_ip_for_outbound_sockets=127.16.224.131
--tserver_master_addrs=127.16.224.190:44171
--webserver_port=43361
--webserver_interface=127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:35.493711 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:35.454877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:52353 (local address 127.16.224.190:44171)
0809 16:13:35.454934 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:35.454936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:35.471116 (+ 16180us) server_negotiation.cc:407] Connection header received
0809 16:13:35.474846 (+ 3730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:35.474849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:35.474867 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:35.474949 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:35.477445 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.478260 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.481009 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.481341 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.481380 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:35.485134 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:35.485135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:35.485136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:35.485144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:35.486851 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.492538 (+ 5687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.492540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.492540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.492624 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.492925 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.492926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.492926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.493078 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:35.493129 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:35.493492 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 16:13:35.493612 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:13:35.494287 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:35.454961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44171 (local address 127.16.224.130:52353)
0809 16:13:35.470923 (+ 15962us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:35.470995 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:35.474697 (+ 3702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:35.476749 (+ 2052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:35.476755 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:35.477015 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:35.477322 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:35.477329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.479567 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.479568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:35.480905 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:35.480908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.481500 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.481501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:35.481653 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:35.482294 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:35.482306 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:35.485018 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:35.489221 (+ 4203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:35.489229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:35.489232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:35.489552 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:35.492699 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:35.492699 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:35.492700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:35.492852 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:35.493160 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:35.493166 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:35.493338 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 16:13:35.493880 (+ 542us) client_negotiation.cc:240] Negotiation successful
0809 16:13:35.494057 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15440,"thread_start_us":152,"threads_started":1}
I20260809 16:13:35.494757 6145 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44171
I20260809 16:13:35.494859 6145 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.495067 6145 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
W20260809 16:13:35.495843 6145 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:44171 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:13:35.495918 6145 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.495980 6145 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:35.496261 6145 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.496349 6145 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
W20260809 16:13:35.496608 6145 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:13:35.569360 6148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:35.569645 6148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:35.569680 6148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:35.571398 6148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:35.571465 6148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:35.571506 6148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:13:35.573660 6148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.224.148:41643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:38219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=43361
--tserver_master_addrs=127.16.224.190:44171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:35.573953 6148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:35.574235 6148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:35.580798 6154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:35.580839 6155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:35.581094 6159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:35.581540 6148 server_base.cc:1034] running on GCE node
I20260809 16:13:35.581751 6148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:35.581996 6148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:35.583168 6148 hybrid_clock.cc:648] HybridClock initialized: now 1786292015583142 us; error 36 us; skew 500 ppm
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:35.585883 6148 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:13:35.586380 6148 webserver.cc:460] Webserver started at http://127.16.224.131:43361/ using document root <none> and password file <none>
I20260809 16:13:35.586618 6148 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:35.586680 6148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:35.600862 6148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:35.602347 6172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.602530 6148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:13:35.602634 6148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "baba9f7916a841299d65965e548d01ee"
format_stamp: "Formatted at 2026-08-09 16:13:34 on dist-test-slave-fgb3"
server_key: "6556fb7040bb5a4b981d40cad40937f6"
server_key_iv: "4c5d384f8078a2444ad7d328d3f93d8e"
server_key_version: "kuduclusterkey@0"
I20260809 16:13:35.603178 6148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:35.616614 6148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:35.619204 6148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:35.619613 6148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:35.619917 6148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:35.620311 6148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:35.620361 6148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.620394 6148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:35.620415 6148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:35.628448 6148 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:38219
I20260809 16:13:35.628625 6321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:38219 every 8 connection(s)
I20260809 16:13:35.628927 6148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:13:35.629474 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 6148
Aug 09 16:13:35 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292015, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:35.641494 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:35.630119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:58587 (local address 127.16.224.190:44171)
0809 16:13:35.630175 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:35.630178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:35.630431 (+ 253us) server_negotiation.cc:407] Connection header received
0809 16:13:35.631199 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:35.631200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:35.631215 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:35.631253 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:35.632143 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.632943 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.634535 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.634862 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.634913 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:35.637800 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:35.637802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:35.637802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:35.637811 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:35.639600 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.640142 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.640144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.640144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.640211 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.640521 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.640522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.640522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.640668 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:35.640735 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:35.641311 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 16:13:35.641399 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:13:35.641510 6325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:35.629788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44171 (local address 127.16.224.131:58587)
0809 16:13:35.630252 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:35.630302 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:35.631049 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:35.631462 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:35.631466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:35.631706 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:35.632011 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:35.632019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.633076 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.633077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:35.634418 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:35.634422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.635044 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.635045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:35.635194 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:35.635780 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:35.635802 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:35.637661 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:35.639704 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:35.639711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:35.639713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:35.640044 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:35.640290 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:35.640291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:35.640291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:35.640436 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:35.640780 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:35.640787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:35.640954 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 16:13:35.641073 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 16:13:35.641262 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":90,"threads_started":1}
I20260809 16:13:35.642097 6323 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44171
I20260809 16:13:35.642210 6323 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.642450 6323 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
W20260809 16:13:35.643136 6323 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:44171 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:13:35.643222 6323 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.643297 6323 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:35.643620 6323 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:35.643707 6323 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
W20260809 16:13:35.644022 6323 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:13:35.648824 6327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:35.635923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50458 (local address 127.16.224.190:44171)
0809 16:13:35.636729 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:35.636731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:35.636966 (+ 235us) server_negotiation.cc:407] Connection header received
0809 16:13:35.636991 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:35.636992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:35.637008 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:35.637073 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:35.641601 (+ 4528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.642397 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.643481 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:35.643772 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:35.643840 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:35.645377 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:35.645378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:35.645378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:35.645386 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:35.647250 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.647645 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.647646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.647647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.647715 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:35.647960 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:35.647961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:35.647961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:35.648105 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:35.648296 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:35.648602 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 16:13:35.648695 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":67,"threads_started":1}
Aug 09 16:13:36 dist-test-slave-fgb3 krb5kdc[4062](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292016, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:36.289162 5955 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:36.289352 5955 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:36.289799 5631 ts_manager.cc:194] Registered new tserver with Master: 0a6afa2c368b48d7a47a0f7682618fc1 (127.16.224.129:33309)
I20260809 16:13:36.290872 5631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:34783
I20260809 16:13:36.496786 6145 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:36.496935 6145 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:36.497182 5631 ts_manager.cc:194] Registered new tserver with Master: 34b5d4ff7f92410493b27ced59601028 (127.16.224.130:39827)
I20260809 16:13:36.497587 5631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:52353
Aug 09 16:13:36 dist-test-slave-fgb3 krb5kdc[4062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292016, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for HTTP/127.16.224.148@KRBTEST.COM
I20260809 16:13:36.644213 6323 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:36.644383 6323 heartbeater.cc:511] Master 127.16.224.190:44171 requested a full tablet report, sending...
I20260809 16:13:36.644753 5631 ts_manager.cc:194] Registered new tserver with Master: baba9f7916a841299d65965e548d01ee (127.16.224.131:38219)
I20260809 16:13:36.645273 5631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:58587
I20260809 16:13:36.647315 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:13:36.656126 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.648335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.16.224.190:44171)
0809 16:13:36.648562 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:36.648564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:36.648575 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:13:36.648639 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:36.648640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:36.648653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:36.648724 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:36.649399 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.650180 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.651241 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.651566 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.651629 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.653107 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:36.653109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:36.653109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:36.653117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:36.654932 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:36.655333 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:36.655334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:36.655335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:36.655405 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:36.655595 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:36.655595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:36.655596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:36.655716 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:36.655807 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:36.655938 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:13:36.656022 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
I20260809 16:13:36.658416 5631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:13:36.660790 5631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:13:36.671059 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.665761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60566 (local address 127.16.224.129:33309)
0809 16:13:36.665903 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:36.665908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:36.666698 (+ 790us) server_negotiation.cc:407] Connection header received
0809 16:13:36.666799 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:36.666803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:36.666819 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:36.666868 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:36.667118 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.668184 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.669741 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.670561 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.670620 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.670675 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:36.670934 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 16:13:36.670978 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260809 16:13:36.671257 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.665339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.16.224.131:38219)
0809 16:13:36.665531 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:36.665537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:36.666555 (+ 1018us) server_negotiation.cc:407] Connection header received
0809 16:13:36.666667 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:36.666670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:36.666687 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:36.666741 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:36.667017 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.668129 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.669749 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.670559 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.670608 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.670665 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:36.671134 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 16:13:36.671175 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":90,"threads_started":1}
I20260809 16:13:36.671571 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.665636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:33309 (local address 127.0.0.1:60566)
0809 16:13:36.666644 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:36.666652 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:36.666722 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:36.666986 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:36.666987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:36.666989 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:36.667045 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.667046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.668286 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.668287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.669637 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.669641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.670734 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.670735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.670812 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.670826 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:13:36.671425 (+ 599us) client_negotiation.cc:240] Negotiation successful
0809 16:13:36.671496 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":5,"thread_start_us":67,"threads_started":1}
I20260809 16:13:36.671684 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.665243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:38219 (local address 127.0.0.1:57594)
0809 16:13:36.666437 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:36.666485 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:36.666588 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:36.666868 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:36.666871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:36.666873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:36.666940 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.666941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.668493 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.668494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.669650 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.669652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.670979 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.670980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.671046 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.671050 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:13:36.671554 (+ 504us) client_negotiation.cc:240] Negotiation successful
0809 16:13:36.671604 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1152,"thread_start_us":288,"threads_started":1}
I20260809 16:13:36.675055 5846 tablet_service.cc:1467] Processing CreateTablet for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c (DEFAULT_TABLE table=test-table [id=7a300c47257d483bb8861105816c89a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:36.675072 6211 tablet_service.cc:1467] Processing CreateTablet for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c (DEFAULT_TABLE table=test-table [id=7a300c47257d483bb8861105816c89a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:36.675421 5846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:36.675472 6211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:36.678406 6371 tablet_bootstrap.cc:492] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1: Bootstrap starting.
I20260809 16:13:36.679049 6371 tablet_bootstrap.cc:654] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:36.679379 6371 log.cc:824] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:36.679754 6375 tablet_bootstrap.cc:492] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Bootstrap starting.
I20260809 16:13:36.680267 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.665251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:39827 (local address 127.0.0.1:37384)
0809 16:13:36.666439 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:36.666485 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:36.666552 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:36.672045 (+ 5493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:36.672047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:36.672049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:36.672089 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.672091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.673166 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.673167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.674290 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.674292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.680024 (+ 5732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.680026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.680100 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.680106 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:13:36.680135 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 16:13:36.680180 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1150,"thread_start_us":179,"threads_started":1}
I20260809 16:13:36.680351 6375 tablet_bootstrap.cc:654] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:36.680653 6375 log.cc:824] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:36.680856 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.665406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.16.224.130:39827)
0809 16:13:36.671888 (+ 6482us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:36.671894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:36.671906 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:13:36.671941 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:36.671943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:36.671960 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:36.672042 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:36.672139 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.673094 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.674457 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.679917 (+ 5460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.680571 (+ 654us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.680655 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:36.680685 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 16:13:36.680776 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6440,"thread_start_us":86,"threads_started":1}
I20260809 16:13:36.681648 6053 tablet_service.cc:1467] Processing CreateTablet for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c (DEFAULT_TABLE table=test-table [id=7a300c47257d483bb8861105816c89a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:36.681769 6371 tablet_bootstrap.cc:492] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1: No bootstrap required, opened a new log
I20260809 16:13:36.681835 6371 ts_tablet_manager.cc:1397] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:13:36.681931 6375 tablet_bootstrap.cc:492] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: No bootstrap required, opened a new log
I20260809 16:13:36.681988 6375 ts_tablet_manager.cc:1397] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:13:36.682018 6053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:36.684358 6371 raft_consensus.cc:348] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.684530 6371 raft_consensus.cc:374] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:36.684428 6375 raft_consensus.cc:348] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.684561 6371 raft_consensus.cc:729] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a6afa2c368b48d7a47a0f7682618fc1, State: Initialized, Role: FOLLOWER
I20260809 16:13:36.684574 6375 raft_consensus.cc:374] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:36.684605 6375 raft_consensus.cc:729] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baba9f7916a841299d65965e548d01ee, State: Initialized, Role: FOLLOWER
I20260809 16:13:36.684677 6371 consensus_queue.cc:260] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.684715 6375 consensus_queue.cc:260] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.684970 6375 ts_tablet_manager.cc:1428] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:13:36.685104 6323 heartbeater.cc:503] Master 127.16.224.190:44171 was elected leader, sending a full tablet report...
I20260809 16:13:36.684996 6371 ts_tablet_manager.cc:1428] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:13:36.686084 5955 heartbeater.cc:503] Master 127.16.224.190:44171 was elected leader, sending a full tablet report...
I20260809 16:13:36.690529 6381 tablet_bootstrap.cc:492] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Bootstrap starting.
I20260809 16:13:36.691228 6381 tablet_bootstrap.cc:654] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:36.691546 6381 log.cc:824] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:36.692363 6381 tablet_bootstrap.cc:492] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: No bootstrap required, opened a new log
I20260809 16:13:36.692462 6381 ts_tablet_manager.cc:1397] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:36.695075 6381 raft_consensus.cc:348] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.695257 6381 raft_consensus.cc:374] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:36.695288 6381 raft_consensus.cc:729] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b5d4ff7f92410493b27ced59601028, State: Initialized, Role: FOLLOWER
I20260809 16:13:36.695448 6381 consensus_queue.cc:260] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.695739 6381 ts_tablet_manager.cc:1428] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:13:36.695848 6145 heartbeater.cc:503] Master 127.16.224.190:44171 was elected leader, sending a full tablet report...
I20260809 16:13:36.697336 6380 raft_consensus.cc:482] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:36.697456 6380 raft_consensus.cc:504] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.697832 6380 leader_election.cc:288] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baba9f7916a841299d65965e548d01ee (127.16.224.131:38219), 34b5d4ff7f92410493b27ced59601028 (127.16.224.130:39827)
I20260809 16:13:36.701856 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.698375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:41049 (local address 127.16.224.131:38219)
0809 16:13:36.698424 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:36.698426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:36.698543 (+ 117us) server_negotiation.cc:407] Connection header received
0809 16:13:36.698603 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:36.698604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:36.698614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:36.698669 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:36.698878 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.699518 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.700947 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.701485 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.701529 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.701538 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:36.701745 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 16:13:36.701786 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:13:36.702008 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.698290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:33191 (local address 127.16.224.130:39827)
0809 16:13:36.698351 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:36.698353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:36.698673 (+ 320us) server_negotiation.cc:407] Connection header received
0809 16:13:36.698755 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:36.698756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:36.698765 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:36.698792 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:36.698989 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.699723 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.701083 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.701641 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.701688 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.701696 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:36.701905 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:13:36.701947 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:13:36.702329 6385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.698203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:39827 (local address 127.16.224.129:33191)
0809 16:13:36.698625 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:36.698632 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:36.698693 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:36.698849 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:36.698850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:36.698851 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:36.698923 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.698924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.699812 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.699813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.700985 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.700988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.701753 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.701754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.701836 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.701839 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:13:36.702173 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 16:13:36.702219 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":60,"threads_started":1}
I20260809 16:13:36.702534 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.697976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:38219 (local address 127.16.224.129:41049)
0809 16:13:36.698473 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:36.698484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:36.698548 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:36.698726 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:36.698727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:36.698729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:36.698806 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.698808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.699613 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.699614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.700837 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:36.700840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.701611 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.701612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:36.701674 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.701678 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:13:36.702254 (+ 576us) client_negotiation.cc:240] Negotiation successful
0809 16:13:36.702300 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":34,"thread_start_us":103,"threads_started":1}
I20260809 16:13:36.702764 6078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f20d91bad0d48ad9b07a1b4d6d7c97c" candidate_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b5d4ff7f92410493b27ced59601028" is_pre_election: true
I20260809 16:13:36.702910 6078 raft_consensus.cc:2454] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a6afa2c368b48d7a47a0f7682618fc1 in term 0.
I20260809 16:13:36.703166 5793 leader_election.cc:302] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a6afa2c368b48d7a47a0f7682618fc1, 34b5d4ff7f92410493b27ced59601028; no voters:
I20260809 16:13:36.703370 6380 raft_consensus.cc:2781] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:13:36.703455 6380 raft_consensus.cc:482] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 16:13:36.703445 6146 tablet.cc:2369] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:13:36.703487 6380 raft_consensus.cc:3037] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:36.704402 6380 raft_consensus.cc:504] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.704545 6380 leader_election.cc:288] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [CANDIDATE]: Term 1 election: Requested vote from peers baba9f7916a841299d65965e548d01ee (127.16.224.131:38219), 34b5d4ff7f92410493b27ced59601028 (127.16.224.130:39827)
I20260809 16:13:36.704823 6262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f20d91bad0d48ad9b07a1b4d6d7c97c" candidate_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baba9f7916a841299d65965e548d01ee" is_pre_election: true
I20260809 16:13:36.704969 6262 raft_consensus.cc:2454] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a6afa2c368b48d7a47a0f7682618fc1 in term 0.
I20260809 16:13:36.705188 6260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f20d91bad0d48ad9b07a1b4d6d7c97c" candidate_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baba9f7916a841299d65965e548d01ee"
I20260809 16:13:36.705250 6260 raft_consensus.cc:3037] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:36.706084 6260 raft_consensus.cc:2454] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a6afa2c368b48d7a47a0f7682618fc1 in term 1.
I20260809 16:13:36.706358 5790 leader_election.cc:302] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a6afa2c368b48d7a47a0f7682618fc1, baba9f7916a841299d65965e548d01ee; no voters:
I20260809 16:13:36.706308 6078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f20d91bad0d48ad9b07a1b4d6d7c97c" candidate_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b5d4ff7f92410493b27ced59601028"
I20260809 16:13:36.706420 6078 raft_consensus.cc:3037] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:36.706609 6380 raft_consensus.cc:2781] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:13:36.706779 6380 raft_consensus.cc:686] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [term 1 LEADER]: Becoming Leader. State: Replica: 0a6afa2c368b48d7a47a0f7682618fc1, State: Running, Role: LEADER
I20260809 16:13:36.706872 6380 consensus_queue.cc:237] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } }
I20260809 16:13:36.707247 6078 raft_consensus.cc:2454] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a6afa2c368b48d7a47a0f7682618fc1 in term 1.
I20260809 16:13:36.707872 5631 catalog_manager.cc:5495] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a6afa2c368b48d7a47a0f7682618fc1 (127.16.224.129). New cstate: current_term: 1 leader_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a6afa2c368b48d7a47a0f7682618fc1" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 } health_report { overall_health: UNKNOWN } } }
I20260809 16:13:36.720350 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.717137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.16.224.129:33309)
0809 16:13:36.717179 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:36.717181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:36.717190 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:13:36.717278 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:36.717279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:36.717298 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:36.717362 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:13:36.717540 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.718259 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.719399 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:36.719727 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:36.719801 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:36.719996 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:13:36.720134 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:13:36.720223 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:36.720248 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 16:13:36.720270 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:13:36.722538 6260 raft_consensus.cc:1261] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 1 FOLLOWER]: Refusing update from remote peer 0a6afa2c368b48d7a47a0f7682618fc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:36.722568 6078 raft_consensus.cc:1261] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 1 FOLLOWER]: Refusing update from remote peer 0a6afa2c368b48d7a47a0f7682618fc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:36.722892 6380 consensus_queue.cc:1035] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baba9f7916a841299d65965e548d01ee" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:36.723340 6380 consensus_queue.cc:1035] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 0a6afa2c368b48d7a47a0f7682618fc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b5d4ff7f92410493b27ced59601028" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 39827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:36.725803 6393 mvcc.cc:205] Tried to move back new op lower bound from 7316652100492361728 to 7316652100432105472. Current Snapshot: MvccSnapshot[applied={T|T < 7316652100492361728}]
I20260809 16:13:36.726675 6396 mvcc.cc:205] Tried to move back new op lower bound from 7316652100492361728 to 7316652100432105472. Current Snapshot: MvccSnapshot[applied={T|T < 7316652100492361728}]
I20260809 16:13:36.728559 6399 mvcc.cc:205] Tried to move back new op lower bound from 7316652100492361728 to 7316652100432105472. Current Snapshot: MvccSnapshot[applied={T|T < 7316652100492361728}]
I20260809 16:13:36.729004 5631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:13:36.729094 5631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:13:36.729943 5631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58bb589e4a194f25a1ac08a8bc629c8a: Sending DeleteTablet for 3 replicas of tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c
I20260809 16:13:36.730374 6053 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:36 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:37384
I20260809 16:13:36.730556 6211 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:36 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:57594
I20260809 16:13:36.730583 5846 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:36 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:60566
I20260809 16:13:36.730787 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 5704
I20260809 16:13:36.746466 6408 tablet_replica.cc:330] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: stopping tablet replica
I20260809 16:13:36.746649 6408 raft_consensus.cc:2229] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:13:36.746942 6408 raft_consensus.cc:2258] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:36.747566 6408 ts_tablet_manager.cc:1905] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:13:36.749290 6408 ts_tablet_manager.cc:1918] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:13:36.749436 6408 log.cc:1196] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/6f20d91bad0d48ad9b07a1b4d6d7c97c
I20260809 16:13:36.749795 6408 ts_tablet_manager.cc:1939] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P baba9f7916a841299d65965e548d01ee: Deleting consensus metadata
I20260809 16:13:36.750349 5610 catalog_manager.cc:4841] TS baba9f7916a841299d65965e548d01ee (127.16.224.131:38219): tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c (table test-table [id=7a300c47257d483bb8861105816c89a8]) successfully deleted
I20260809 16:13:36.750437 6407 tablet_replica.cc:330] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: stopping tablet replica
I20260809 16:13:36.750582 6407 raft_consensus.cc:2229] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:13:36.750694 6407 raft_consensus.cc:2258] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:36.752751 6407 ts_tablet_manager.cc:1905] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:13:36.754251 6407 ts_tablet_manager.cc:1918] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:13:36.754341 6407 log.cc:1196] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/6f20d91bad0d48ad9b07a1b4d6d7c97c
I20260809 16:13:36.754590 6407 ts_tablet_manager.cc:1939] T 6f20d91bad0d48ad9b07a1b4d6d7c97c P 34b5d4ff7f92410493b27ced59601028: Deleting consensus metadata
I20260809 16:13:36.755041 5611 catalog_manager.cc:4841] TS 34b5d4ff7f92410493b27ced59601028 (127.16.224.130:39827): tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c (table test-table [id=7a300c47257d483bb8861105816c89a8]) successfully deleted
W20260809 16:13:36.755373 5610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:33309 (error 108)
I20260809 16:13:36.755525 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 5957
I20260809 16:13:36.758014 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:36.755770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:33309 (local address 127.0.0.1:60594)
0809 16:13:36.757898 (+ 2128us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:36.757955 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2102}
W20260809 16:13:36.758158 5610 catalog_manager.cc:4584] TS 0a6afa2c368b48d7a47a0f7682618fc1 (127.16.224.129:33309): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f20d91bad0d48ad9b07a1b4d6d7c97c: Network error: Client connection negotiation failed: client connection to 127.16.224.129:33309: connect: Connection refused (error 111)
I20260809 16:13:36.768406 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 6148
I20260809 16:13:36.805516 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 5578
I20260809 16:13:36.823921 17282 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:13:37.168316 17282 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:13:37.168416 17282 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:13:37.511826 17282 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:13:37.511 UTC [4111] LOG: received smart shutdown request
2026-08-09 16:13:37.515 UTC [4111] LOG: background worker "logical replication launcher" (PID 4125) exited with exit code 1
2026-08-09 16:13:37.516 UTC [4120] LOG: shutting down
2026-08-09 16:13:37.661 UTC [4111] LOG: database system is shut down
2026-08-09T16:13:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47246 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:13:37 dist-test-slave-fgb3 krb5kdc[6424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:13:37 dist-test-slave-fgb3 krb5kdc[6424](info): set up 2 sockets
Aug 09 16:13:37 dist-test-slave-fgb3 krb5kdc[6424](info): commencing operation
krb5kdc: starting...
W20260809 16:13:38.873226 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:13:38 dist-test-slave-fgb3 krb5kdc[6424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:13:38.987509 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40509
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:40509,127.16.224.189:41393,127.16.224.188:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:39.069618 6439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:39.069835 6439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:39.069859 6439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:39.070999 6439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:39.071040 6439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:39.071053 6439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:39.071066 6439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:39.071090 6439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:39.072619 6439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:40509,127.16.224.189:41393,127.16.224.188:42599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:39.072873 6439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:39.073094 6439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:39.077778 6450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:39.077788 6447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:39.077788 6455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:39.077913 6439 server_base.cc:1034] running on GCE node
I20260809 16:13:39.078269 6439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:39.078572 6439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:39.079710 6439 hybrid_clock.cc:648] HybridClock initialized: now 1786292019079694 us; error 33 us; skew 500 ppm
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:39.085474 6439 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:13:39.085878 6439 webserver.cc:460] Webserver started at http://127.16.224.190:37215/ using document root <none> and password file <none>
I20260809 16:13:39.086072 6439 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:39.086122 6439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:39.086247 6439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:39.087301 6439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "7ab32a50b3fb4b78acb227822cd26777"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "1c947fbcebc761e014222469a54c50a2"
server_key_iv: "199ac152a4a2cd21d4c1ee411efe70aa"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.087647 6439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "7ab32a50b3fb4b78acb227822cd26777"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "1c947fbcebc761e014222469a54c50a2"
server_key_iv: "199ac152a4a2cd21d4c1ee411efe70aa"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.088887 6439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:13:39.089546 6470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:39.089711 6439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:39.089799 6439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "7ab32a50b3fb4b78acb227822cd26777"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "1c947fbcebc761e014222469a54c50a2"
server_key_iv: "199ac152a4a2cd21d4c1ee411efe70aa"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.089859 6439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:39.119228 6439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:39.121174 6439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:39.121313 6439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:39.125221 6439 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:40509
I20260809 16:13:39.125270 6551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:40509 every 8 connection(s)
I20260809 16:13:39.125609 6439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:13:39.126508 6553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:39.128095 6553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:13:39.133195 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 6439
I20260809 16:13:39.133309 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:13:39.133533 17282 external_mini_cluster.cc:1428] Setting key 36be5596c1ed4bca3e080e438f667a88
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:39.137382 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.128745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:40509 (local address 127.0.0.1:54874)
0809 16:13:39.128907 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.128947 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.129390 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.129672 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.129674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.129925 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.130087 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.130091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.131119 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.131119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.131833 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.131835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.132337 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.132337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.132408 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.132860 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.132868 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.134430 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.136037 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.136044 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.136046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.136227 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.136444 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.136445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.136445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.136526 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.136848 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.136853 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.136917 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.136999 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.137208 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":77,"thread_start_us":74,"threads_started":1}
I20260809 16:13:39.137383 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.128776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.16.224.190:40509)
0809 16:13:39.128916 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.128925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.129019 (+ 94us) server_negotiation.cc:407] Connection header received
0809 16:13:39.129555 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.129557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.129575 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.129661 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.130174 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.131034 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.131908 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.132228 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.132372 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.134511 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.134518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.134519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.134525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.135958 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.136306 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.136308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.136308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.136384 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.136579 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.136580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.136581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.136783 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.136855 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.137031 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.137137 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260809 16:13:39.138286 6553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:13:39.138674 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.138552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59068)
0809 16:13:39.138598 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.138648 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:13:39.138816 6475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
I20260809 16:13:39.139227 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.139131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59082)
0809 16:13:39.139173 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.139204 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:13:39.139346 6553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:41393: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.16.224.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab.
Entry for principal kudu/127.16.224.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab.
I20260809 16:13:39.157711 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:41393
--webserver_interface=127.16.224.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab
--principal=kudu/127.16.224.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:40509,127.16.224.189:41393,127.16.224.188:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:39.188567 6553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } attempt: 1
I20260809 16:13:39.189055 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.188915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59088)
0809 16:13:39.188976 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.189019 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:13:39.189396 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.189298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59100)
0809 16:13:39.189341 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.189371 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:13:39.189495 6553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:41393: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
W20260809 16:13:39.239291 6566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:39.239485 6566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:39.239506 6566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:39.240566 6566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:39.240600 6566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:39.240612 6566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:39.240623 6566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:39.240638 6566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:39.242285 6566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:40509,127.16.224.189:41393,127.16.224.188:42599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab
--principal=kudu/127.16.224.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:41393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.224.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:39.242512 6566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:39.242681 6566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:39.247313 6580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:39.247319 6581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:39.247319 6583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:39.247426 6566 server_base.cc:1034] running on GCE node
I20260809 16:13:39.247762 6566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:39.248030 6566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:39.249181 6566 hybrid_clock.cc:648] HybridClock initialized: now 1786292019249168 us; error 28 us; skew 500 ppm
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:39.251580 6566 init.cc:377] Logged in from keytab as kudu/127.16.224.189@KRBTEST.COM (short username kudu)
I20260809 16:13:39.251984 6566 webserver.cc:460] Webserver started at http://127.16.224.189:36279/ using document root <none> and password file <none>
I20260809 16:13:39.252184 6566 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:39.252223 6566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:39.252323 6566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:39.253326 6566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data/instance:
uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "053e86596bd688c16bf49fd1d4c905e0"
server_key_iv: "40d328073537b9d75913a71cec618bb3"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.253625 6566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/wal/instance:
uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "053e86596bd688c16bf49fd1d4c905e0"
server_key_iv: "40d328073537b9d75913a71cec618bb3"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.254876 6566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:13:39.255509 6596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:39.255672 6566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:13:39.255754 6566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/wal
uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "053e86596bd688c16bf49fd1d4c905e0"
server_key_iv: "40d328073537b9d75913a71cec618bb3"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.255807 6566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:39.257627 6553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } attempt: 2
I20260809 16:13:39.258016 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.257863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59110)
0809 16:13:39.257923 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.257966 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:13:39.258334 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.258225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59116)
0809 16:13:39.258263 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.258290 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 16:13:39.258410 6553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:41393: Network error: Client connection negotiation failed: client connection to 127.16.224.189:41393: connect: Connection refused (error 111)
I20260809 16:13:39.274986 6566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:39.277097 6566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:39.277215 6566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:39.281080 6566 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.189:41393
I20260809 16:13:39.281127 6673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.189:41393 every 8 connection(s)
I20260809 16:13:39.281456 6566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
I20260809 16:13:39.282402 6675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:39.282907 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 6566
I20260809 16:13:39.283005 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-1/wal/instance
I20260809 16:13:39.283186 17282 external_mini_cluster.cc:1428] Setting key 2f14ac7341fca2eb41deb5fbfee32fca
I20260809 16:13:39.284102 6675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.224.188@KRBTEST.COM; defaulting to no policy
I20260809 16:13:39.293875 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.284858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54878 (local address 127.16.224.190:40509)
0809 16:13:39.284921 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.284923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.285161 (+ 238us) server_negotiation.cc:407] Connection header received
0809 16:13:39.285642 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.285644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.285659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.285721 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.286340 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.286873 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.288307 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.288530 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.288587 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.291063 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.291065 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.291065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.291073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.292457 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.292839 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.292841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.292841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.292913 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.293243 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.293244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.293245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.293363 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.293419 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.293723 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.293800 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:13:39.294000 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.284848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:40509 (local address 127.0.0.1:54878)
0809 16:13:39.285035 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.285074 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.285553 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.285791 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.285794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.286044 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.286243 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.286248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.286982 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.286983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.288213 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.288216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.288615 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.288616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.288765 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.289281 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.289291 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.290966 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.292538 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.292546 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.292548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.292755 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.293015 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.293017 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.293017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.293171 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.293440 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.293447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.293603 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.293695 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.293817 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.188@KRBTEST.COM" created.
I20260809 16:13:39.294994 6675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:13:39.303041 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.295336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59120 (local address 127.16.224.189:41393)
0809 16:13:39.296534 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.296538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.296557 (+ 19us) server_negotiation.cc:407] Connection header received
0809 16:13:39.296576 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.296578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.296589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.296649 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.296936 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.297601 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.298545 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.298837 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.298897 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.300610 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.300618 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.300618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.300624 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.301976 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.302342 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.302346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.302346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.302398 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.302577 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.302578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.302578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.302701 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.302756 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.302901 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.302967 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169,"thread_start_us":70,"threads_started":1}
I20260809 16:13:39.303102 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.295280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59120)
0809 16:13:39.295326 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.295337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.295376 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.296668 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.296669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.296805 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.296869 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.296871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.297683 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.297684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.298464 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.298467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.298905 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.298906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.298968 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.299370 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.299374 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.300531 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.302058 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.302059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.302060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.302251 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.302462 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.302463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.302463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.302524 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.302757 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.302757 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.302803 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.302878 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.303020 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":59}
I20260809 16:13:39.303776 6675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:13:39.304085 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.303971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:48962)
0809 16:13:39.304019 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.304066 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:13:39.304205 6598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
I20260809 16:13:39.304597 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.304516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:48964)
0809 16:13:39.304556 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.304580 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:13:39.304701 6675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:42599: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab.
Entry for principal kudu/127.16.224.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab.
I20260809 16:13:39.308910 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:42599
--webserver_interface=127.16.224.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab
--principal=kudu/127.16.224.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:40509,127.16.224.189:41393,127.16.224.188:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:39.349560 6553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } attempt: 3
I20260809 16:13:39.353910 6675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } attempt: 1
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:13:39.354300 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.354171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:48970)
0809 16:13:39.354224 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.354258 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:13:39.354666 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.354571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:48980)
0809 16:13:39.354614 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.354641 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:13:39.354760 6675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:42599: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
I20260809 16:13:39.356952 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.350089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.16.224.189:41393)
0809 16:13:39.350160 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.350163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.350176 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:13:39.350227 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.350229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.350243 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.350299 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.350639 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.351411 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.352114 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.352447 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.352520 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.354647 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.354648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.354648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.354655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.356002 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.356358 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.356358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.356359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.356405 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.356570 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.356570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.356571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.356665 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.356710 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.356845 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.356901 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:13:39.357015 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.349994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59130)
0809 16:13:39.350074 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.350087 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.350157 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.350347 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.350348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.350501 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.350573 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.350574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.351503 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.351504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.352039 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.352041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.352544 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.352545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.352640 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.353092 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.353097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.354574 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.356081 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.356082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.356083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.356294 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.356460 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.356461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.356461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.356520 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.356712 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.356713 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.356762 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.356828 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.356922 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 16:13:39.357429 6553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:13:39.357774 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.357668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:48990)
0809 16:13:39.357717 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.357748 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:13:39.358091 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.358007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49002)
0809 16:13:39.358046 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.358072 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:13:39.358162 6553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:42599: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
I20260809 16:13:39.389324 6553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } attempt: 1
I20260809 16:13:39.389820 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.389661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49014)
0809 16:13:39.389726 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.389783 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:13:39.390123 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.390040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49022)
0809 16:13:39.390076 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.390104 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 16:13:39.390221 6553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:42599: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
W20260809 16:13:39.395247 6685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:39.395440 6685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:39.395462 6685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:39.396543 6685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:39.396582 6685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:39.396595 6685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:39.396607 6685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:39.396629 6685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:39.398123 6685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:40509,127.16.224.189:41393,127.16.224.188:42599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab
--principal=kudu/127.16.224.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.224.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:39.398384 6685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:39.398591 6685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:39.403126 6693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:39.403151 6696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:39.403151 6691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:39.403364 6685 server_base.cc:1034] running on GCE node
I20260809 16:13:39.403597 6685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:39.403851 6685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:39.404987 6685 hybrid_clock.cc:648] HybridClock initialized: now 1786292019404978 us; error 30 us; skew 500 ppm
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:39.407320 6685 init.cc:377] Logged in from keytab as kudu/127.16.224.188@KRBTEST.COM (short username kudu)
I20260809 16:13:39.407727 6685 webserver.cc:460] Webserver started at http://127.16.224.188:41769/ using document root <none> and password file <none>
I20260809 16:13:39.407902 6685 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:39.407939 6685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:39.408030 6685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:39.408977 6685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data/instance:
uuid: "900f9cb5352840f1aab5ace3e3515e26"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "53557a245154e0f58d142ed9c6ce01cf"
server_key_iv: "021633614a94fcbeb6ce043794561294"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.409257 6685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/wal/instance:
uuid: "900f9cb5352840f1aab5ace3e3515e26"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "53557a245154e0f58d142ed9c6ce01cf"
server_key_iv: "021633614a94fcbeb6ce043794561294"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.410419 6685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:13:39.411103 6710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:39.411293 6685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:13:39.411386 6685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/wal
uuid: "900f9cb5352840f1aab5ace3e3515e26"
format_stamp: "Formatted at 2026-08-09 16:13:39 on dist-test-slave-fgb3"
server_key: "53557a245154e0f58d142ed9c6ce01cf"
server_key_iv: "021633614a94fcbeb6ce043794561294"
server_key_version: "encryptionkey@0"
I20260809 16:13:39.411458 6685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:39.422878 6685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:39.422914 6675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } attempt: 2
I20260809 16:13:39.423381 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.423184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49024)
0809 16:13:39.423302 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.423344 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 16:13:39.423769 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.423673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49032)
0809 16:13:39.423718 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.423746 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:13:39.423868 6675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:42599: Network error: Client connection negotiation failed: client connection to 127.16.224.188:42599: connect: Connection refused (error 111)
I20260809 16:13:39.424930 6685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:39.425063 6685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:39.428726 6685 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.188:42599
I20260809 16:13:39.428768 6789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.188:42599 every 8 connection(s)
I20260809 16:13:39.429085 6685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
I20260809 16:13:39.429909 6791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:39.431448 6791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:13:39.433776 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 6685
I20260809 16:13:39.433892 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786291868496945-17282-0/minicluster-data/master-2/wal/instance
I20260809 16:13:39.434070 17282 external_mini_cluster.cc:1428] Setting key 797f500e7b7ecadfa73e04f3ece42be5
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:13:39.440527 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.432131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.16.224.190:40509)
0809 16:13:39.432197 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.432199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.432360 (+ 161us) server_negotiation.cc:407] Connection header received
0809 16:13:39.432819 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.432821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.432834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.432886 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.433502 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.434008 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.435327 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.435554 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.435646 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.437815 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.437816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.437816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.437825 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.439210 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.439580 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.439582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.439582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.439651 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.439872 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.439873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.439874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.440018 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.440078 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.440380 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.440463 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:13:39.440709 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.432078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:40509 (local address 127.0.0.1:54892)
0809 16:13:39.432255 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.432290 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.432726 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.432977 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.432980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.433212 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.433407 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.433411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.434093 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.434094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.435231 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.435234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.435646 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.435646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.435761 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.436178 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.436187 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.437727 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.439293 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.439299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.439300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.439506 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.439716 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.439716 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.439716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.439809 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.440088 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.440094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.440245 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.440344 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.440497 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260809 16:13:39.441236 6791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:13:39.442663 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.434458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.16.224.188:42599)
0809 16:13:39.434625 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.434629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.434808 (+ 179us) server_negotiation.cc:407] Connection header received
0809 16:13:39.434834 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.434837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.434858 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.434926 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.435589 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.436326 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.437158 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.437587 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.437668 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.439979 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.439983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.439983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.439988 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.441601 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.441880 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.441883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.441884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.441937 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.442097 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.442097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.442098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.442225 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.442385 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.442495 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.442565 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:39.447620 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.441527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.16.224.189:41393)
0809 16:13:39.441571 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.441572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.441581 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:13:39.441607 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.441608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.441616 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.441670 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.441923 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.442420 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.443199 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.443453 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.443514 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.445278 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.445280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.445280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.445287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.446584 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.446958 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.446959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.446960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.447018 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.447183 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.447183 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.447184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.447302 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.447363 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.447504 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.447563 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:13:39.447657 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.441458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:41393 (local address 127.0.0.1:59138)
0809 16:13:39.441504 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.441515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.441556 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.441685 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.441686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.441820 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.441848 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.441850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.442486 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.442487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.443109 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.443111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.443522 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.443524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.443582 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.443956 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.443961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.445199 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.446658 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.446660 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.446660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.446876 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.447083 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.447083 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.447084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.447133 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.447358 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.447359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.447402 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.447482 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.447567 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:13:39.448315 6791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:13:39.450261 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.443398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.16.224.190:40509)
0809 16:13:39.443434 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.443436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.443455 (+ 19us) server_negotiation.cc:407] Connection header received
0809 16:13:39.443479 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.443480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.443490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.443549 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.443977 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.444597 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.445359 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.445534 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.445595 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.447792 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.447793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.447794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.447801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.449300 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.449633 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.449634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.449635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.449685 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.449872 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.449872 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.449872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.449960 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.450070 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.450140 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.450193 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:13:39.454289 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.448543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49048)
0809 16:13:39.448586 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.448595 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.448629 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.448790 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.448791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.448925 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.448959 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.448961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.449523 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.449524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.450187 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.450190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.450631 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.450632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.450690 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.451041 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.451045 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.451995 (+ 950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.453405 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.453406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.453406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.453554 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.453718 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.453719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.453719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.453768 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.454035 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.454036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.454075 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.454131 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.454201 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:13:39.454391 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.448591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49048 (local address 127.16.224.188:42599)
0809 16:13:39.448691 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.448693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.448699 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:13:39.448720 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.448721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.448728 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.448768 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.449028 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.449444 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.450273 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.450558 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.450617 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.452063 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.452064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.452064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.452070 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.453338 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.453616 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.453617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.453617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.453669 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.453817 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.453818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.453818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.453940 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.454014 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.454166 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.454253 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":84}
I20260809 16:13:39.455775 6791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: Bootstrap starting.
I20260809 16:13:39.456691 6791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:39.457005 6791 log.cc:824] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:39.457165 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.450847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59140 (local address 127.16.224.189:41393)
0809 16:13:39.450890 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.450892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.450901 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:13:39.450933 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.450934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.450943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.450985 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.451421 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.451867 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.452509 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.452707 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.452773 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.454865 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.454866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.454866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.454873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.456145 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.456495 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.456496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.456496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.456538 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.456704 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.456705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.456705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.456823 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.456955 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.457020 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.457081 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:13:39.457882 6791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: No bootstrap required, opened a new log
I20260809 16:13:39.458932 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:13:39.459615 6791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.459739 6791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:39.459818 6791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900f9cb5352840f1aab5ace3e3515e26, State: Initialized, Role: FOLLOWER
I20260809 16:13:39.459946 6791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.460259 6791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:39.460216 6806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.460330 6806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:13:39.462016 6822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:13:39.462261 6822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:13:39.465379 6553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } attempt: 2
I20260809 16:13:39.462687 6685 master_runner.cc:186] Error getting master registration for 127.16.224.190:40509: OK, instance_id { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" instance_seqno: 1786292019124089 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:13:39.467079 6685 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:13:39.467973 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.460390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59154 (local address 127.16.224.189:41393)
0809 16:13:39.460432 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.460433 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.460458 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:13:39.460522 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.460523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.460532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.460575 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.461172 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.461818 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.463612 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.463863 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.463928 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.465122 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.465123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.465123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.465129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.466777 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.467139 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.467140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.467140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.467195 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.467424 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.467425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.467425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.467554 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.467630 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.467810 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.467884 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:13:39.469136 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.460187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.16.224.190:40509)
0809 16:13:39.460223 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.460225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.460590 (+ 365us) server_negotiation.cc:407] Connection header received
0809 16:13:39.460666 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.460667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.460676 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.460729 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.461300 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.461959 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.463105 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.463406 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.463460 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.464826 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.464828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.464828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.464835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.466400 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.467281 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.467282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.467283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.467335 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.467535 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.467536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.467536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.467661 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.467737 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.468987 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.469056 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:13:39.472869 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.461730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.16.224.188:42599)
0809 16:13:39.461758 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.461759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.461985 (+ 226us) server_negotiation.cc:407] Connection header received
0809 16:13:39.462069 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.462070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.462078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.462104 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.466999 (+ 4895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.467622 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.468536 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.468786 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.468849 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.470257 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.470259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.470259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.470265 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.471687 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.472037 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.472038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.472038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.472082 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.472280 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.472282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.472282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.472435 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.472538 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.472708 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.472786 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 16:13:39.475169 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.465583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49060)
0809 16:13:39.465628 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.465637 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.465671 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.467560 (+ 1889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.467561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.467702 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.467736 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.467737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.468404 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.468406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.468889 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.468891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.470026 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.470027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.470089 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.470482 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.470487 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.471907 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.474115 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.474116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.474116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.474267 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.474504 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.474505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.474506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.474571 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.474834 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.474835 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.474885 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.474972 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.475072 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 16:13:39.475196 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.465631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.16.224.188:42599)
0809 16:13:39.467436 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.467438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.467445 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:13:39.467461 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.467462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.467472 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.467800 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.467839 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.468329 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.469636 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.469940 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.470008 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.471976 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.471977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.471977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.471983 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.474041 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.474346 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.474347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.474347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.474407 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.474643 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.474644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.474644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.474757 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.474815 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.475010 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.475084 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1783,"thread_start_us":85,"threads_started":1}
I20260809 16:13:39.476617 6553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: Bootstrap starting.
I20260809 16:13:39.477232 6553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:39.477506 6553 log.cc:824] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:39.478250 6553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: No bootstrap required, opened a new log
I20260809 16:13:39.479682 6553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.479794 6553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:39.479826 6553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab32a50b3fb4b78acb227822cd26777, State: Initialized, Role: FOLLOWER
I20260809 16:13:39.479941 6553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.480198 6828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.480255 6553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:39.480293 6828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:13:39.481971 6843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:13:39.482216 6843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:13:39.483103 6439 master_runner.cc:186] Error getting master registration for 127.16.224.189:41393: OK, instance_id { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" instance_seqno: 1786292019280020 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:13:39.485323 6439 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:13:39.515004 6675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } attempt: 3
Aug 09 16:13:39 dist-test-slave-fgb3 krb5kdc[6424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292019, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:13:39.523352 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.515334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:42599 (local address 127.0.0.1:49070)
0809 16:13:39.515386 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:39.515398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:39.515445 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:39.515595 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:39.515596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:39.515787 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:39.515848 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.515850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.517038 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.517039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.517866 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:39.517869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.518384 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.518385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:39.518444 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.518933 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:39.518938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:39.520435 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:39.522256 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.522258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.522258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.522498 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.522700 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:39.522701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:39.522701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:39.522772 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:39.523041 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:39.523042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:39.523095 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:13:39.523165 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:13:39.523259 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:13:39.523376 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:39.515398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49070 (local address 127.16.224.188:42599)
0809 16:13:39.515436 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:39.515438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:39.515447 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:13:39.515513 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:39.515514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:39.515525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:39.515580 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:39.515922 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.516689 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.517947 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:39.518282 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:39.518384 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:39.520526 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:39.520528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:39.520528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:39.520536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:39.522128 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.522571 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.522572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.522573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.522635 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:39.522840 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:39.522840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:39.522841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:39.522972 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:39.523037 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:39.523197 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:13:39.523287 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:13:39.525519 6675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3: Bootstrap starting.
I20260809 16:13:39.526389 6675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:39.526726 6675 log.cc:824] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:39.527653 6675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3: No bootstrap required, opened a new log
I20260809 16:13:39.530157 6675 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.530298 6675 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:39.530359 6675 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd18b5062d6a4bd9a4d0698f91bd91f3, State: Initialized, Role: FOLLOWER
I20260809 16:13:39.530483 6675 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.530797 6675 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:39.530766 6846 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.530951 6846 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:13:39.532640 6866 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:13:39.532738 6866 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:13:39.534880 6566 master_runner.cc:263] No leader master found from master bd18b5062d6a4bd9a4d0698f91bd91f3
I20260809 16:13:39.534935 6566 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:13:39.612910 6846 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:39.613027 6846 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.613261 6846 leader_election.cc:288] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ab32a50b3fb4b78acb227822cd26777 (127.16.224.190:40509), 900f9cb5352840f1aab5ace3e3515e26 (127.16.224.188:42599)
I20260809 16:13:39.613534 6518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ab32a50b3fb4b78acb227822cd26777" is_pre_election: true
I20260809 16:13:39.613540 6753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900f9cb5352840f1aab5ace3e3515e26" is_pre_election: true
I20260809 16:13:39.613677 6753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd18b5062d6a4bd9a4d0698f91bd91f3 in term 0.
I20260809 16:13:39.613677 6518 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd18b5062d6a4bd9a4d0698f91bd91f3 in term 0.
I20260809 16:13:39.613870 6597 leader_election.cc:302] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ab32a50b3fb4b78acb227822cd26777, bd18b5062d6a4bd9a4d0698f91bd91f3; no voters:
I20260809 16:13:39.614001 6846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:13:39.614037 6846 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:13:39.614055 6846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:39.614768 6846 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.614884 6846 leader_election.cc:288] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [CANDIDATE]: Term 1 election: Requested vote from peers 7ab32a50b3fb4b78acb227822cd26777 (127.16.224.190:40509), 900f9cb5352840f1aab5ace3e3515e26 (127.16.224.188:42599)
I20260809 16:13:39.615037 6518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ab32a50b3fb4b78acb227822cd26777"
I20260809 16:13:39.615110 6518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:39.615098 6753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900f9cb5352840f1aab5ace3e3515e26"
I20260809 16:13:39.615181 6753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:39.615947 6753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd18b5062d6a4bd9a4d0698f91bd91f3 in term 1.
I20260809 16:13:39.615947 6518 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd18b5062d6a4bd9a4d0698f91bd91f3 in term 1.
I20260809 16:13:39.616112 6597 leader_election.cc:302] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ab32a50b3fb4b78acb227822cd26777, bd18b5062d6a4bd9a4d0698f91bd91f3; no voters:
I20260809 16:13:39.616196 6846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:13:39.616362 6846 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [term 1 LEADER]: Becoming Leader. State: Replica: bd18b5062d6a4bd9a4d0698f91bd91f3, State: Running, Role: LEADER
I20260809 16:13:39.616465 6846 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } }
I20260809 16:13:39.616860 6870 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd18b5062d6a4bd9a4d0698f91bd91f3. Latest consensus state: current_term: 1 leader_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.616940 6870 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:39.617177 6872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:13:39.617379 6872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:13:39.618697 6753 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [term 1 FOLLOWER]: Refusing update from remote peer bd18b5062d6a4bd9a4d0698f91bd91f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:39.618753 6518 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [term 1 FOLLOWER]: Refusing update from remote peer bd18b5062d6a4bd9a4d0698f91bd91f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:39.619099 6846 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:39.619254 6870 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:39.620507 6806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd18b5062d6a4bd9a4d0698f91bd91f3. Latest consensus state: current_term: 1 leader_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.620596 6806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:13:39.620653 6828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd18b5062d6a4bd9a4d0698f91bd91f3. Latest consensus state: current_term: 1 leader_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.620723 6828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:13:39.621714 6870 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.621795 6870 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:39.621763 6828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.621852 6828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:13:39.622058 6479 tablet.cc:2369] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:13:39.622632 6873 mvcc.cc:205] Tried to move back new op lower bound from 7316652112355901440 to 7316652112349593600. Current Snapshot: MvccSnapshot[applied={T|T < 7316652112355901440}]
I20260809 16:13:39.623023 6876 mvcc.cc:205] Tried to move back new op lower bound from 7316652112355901440 to 7316652112349593600. Current Snapshot: MvccSnapshot[applied={T|T < 7316652112355901440}]
I20260809 16:13:39.623476 6872 catalog_manager.cc:1347] Generated new cluster ID: ffda6694369d45a79ec02c218b259a60
I20260809 16:13:39.623565 6872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:13:39.627472 6846 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.627566 6846 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:39.628028 6806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab32a50b3fb4b78acb227822cd26777" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40509 } } peers { permanent_uuid: "bd18b5062d6a4bd9a4d0698f91bd91f3" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 41393 } } peers { permanent_uuid: "900f9cb5352840f1aab5ace3e3515e26" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 42599 } } }
I20260809 16:13:39.628122 6806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:13:39.628343 6875 mvcc.cc:205] Tried to move back new op lower bound from 7316652112355901440 to 7316652112349593600. Current Snapshot: MvccSnapshot[applied={T|T < 7316652112355901440}]
I20260809 16:13:39.638876 6872 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:13:39.639647 6872 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:13:39.646337 6872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd18b5062d6a4bd9a4d0698f91bd91f3: Generated new TSK 0
I20260809 16:13:39.646524 6872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:13:40.462786 6822 catalog_manager.cc:1259] Loaded cluster ID: ffda6694369d45a79ec02c218b259a60
I20260809 16:13:40.462873 6822 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: loading cluster ID for follower catalog manager: success
I20260809 16:13:40.463687 6822 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: acquiring CA information for follower catalog manager: success
I20260809 16:13:40.463986 6822 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 900f9cb5352840f1aab5ace3e3515e26: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:13:40.482754 6843 catalog_manager.cc:1259] Loaded cluster ID: ffda6694369d45a79ec02c218b259a60
I20260809 16:13:40.482824 6843 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: loading cluster ID for follower catalog manager: success
I20260809 16:13:40.483660 6843 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: acquiring CA information for follower catalog manager: success
I20260809 16:13:40.483964 6843 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ab32a50b3fb4b78acb227822cd26777: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:13:40.704588 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 6439
I20260809 16:13:40.718798 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 6566
I20260809 16:13:40.734849 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 6685
2026-08-09T16:13:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2993 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:13:40 dist-test-slave-fgb3 krb5kdc[6900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:13:40 dist-test-slave-fgb3 krb5kdc[6900](info): set up 2 sockets
Aug 09 16:13:40 dist-test-slave-fgb3 krb5kdc[6900](info): commencing operation
krb5kdc: starting...
W20260809 16:13:41.886519 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:13:41 dist-test-slave-fgb3 krb5kdc[6900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:13:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:13:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:13:42.000771 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40765
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:40765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:42.080427 6917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:42.080622 6917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:42.080651 6917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:42.081750 6917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:42.081791 6917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:42.081809 6917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:42.081825 6917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:42.081840 6917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:42.083364 6917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:40765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:42.083612 6917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:42.083834 6917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:42.088341 6925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:42.088408 6927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:42.088337 6930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:42.088459 6917 server_base.cc:1034] running on GCE node
I20260809 16:13:42.088846 6917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:42.089118 6917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:42.090248 6917 hybrid_clock.cc:648] HybridClock initialized: now 1786292022090242 us; error 33 us; skew 500 ppm
Aug 09 16:13:42 dist-test-slave-fgb3 krb5kdc[6900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292022, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:42.092525 6917 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:13:42.092935 6917 webserver.cc:460] Webserver started at http://127.16.224.190:36775/ using document root <none> and password file <none>
I20260809 16:13:42.093107 6917 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:42.093146 6917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:42.093235 6917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:42.094210 6917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "2955c8768961479cb210dd61405175b3"
format_stamp: "Formatted at 2026-08-09 16:13:42 on dist-test-slave-fgb3"
server_key: "01e0117564814eea14b01f6f067da16c"
server_key_iv: "24b7b54d52f67d85cbeea7b788f67e06"
server_key_version: "encryptionkey@0"
I20260809 16:13:42.094579 6917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "2955c8768961479cb210dd61405175b3"
format_stamp: "Formatted at 2026-08-09 16:13:42 on dist-test-slave-fgb3"
server_key: "01e0117564814eea14b01f6f067da16c"
server_key_iv: "24b7b54d52f67d85cbeea7b788f67e06"
server_key_version: "encryptionkey@0"
I20260809 16:13:42.095767 6917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:42.096381 6939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:42.096536 6917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:13:42.096616 6917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "2955c8768961479cb210dd61405175b3"
format_stamp: "Formatted at 2026-08-09 16:13:42 on dist-test-slave-fgb3"
server_key: "01e0117564814eea14b01f6f067da16c"
server_key_iv: "24b7b54d52f67d85cbeea7b788f67e06"
server_key_version: "encryptionkey@0"
I20260809 16:13:42.096679 6917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:42.116580 6917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:42.118722 6917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:42.118853 6917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:42.122874 6917 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:40765
I20260809 16:13:42.122913 7013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:40765 every 8 connection(s)
I20260809 16:13:42.123267 6917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:13:42.124153 7014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:42.124903 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 6917
I20260809 16:13:42.124982 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:13:42.125169 17282 external_mini_cluster.cc:1428] Setting key 2bca3b5f4eab64c03e9a35452c578b46
I20260809 16:13:42.127195 7014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3: Bootstrap starting.
I20260809 16:13:42.127797 7014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:42.128084 7014 log.cc:824] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:42.128731 7014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3: No bootstrap required, opened a new log
I20260809 16:13:42.130996 7014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2955c8768961479cb210dd61405175b3" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40765 } }
I20260809 16:13:42.131186 7014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:42.131223 7014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2955c8768961479cb210dd61405175b3, State: Initialized, Role: FOLLOWER
I20260809 16:13:42.131348 7014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2955c8768961479cb210dd61405175b3" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40765 } }
I20260809 16:13:42.131434 7014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:13:42.131474 7014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:13:42.131520 7014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 16:13:42 dist-test-slave-fgb3 krb5kdc[6900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:42.132095 7014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2955c8768961479cb210dd61405175b3" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40765 } }
I20260809 16:13:42.132210 7014 leader_election.cc:302] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2955c8768961479cb210dd61405175b3; no voters:
I20260809 16:13:42.132383 7014 leader_election.cc:288] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:13:42.132556 7021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:13:42.132668 7014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:42.132741 7021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [term 1 LEADER]: Becoming Leader. State: Replica: 2955c8768961479cb210dd61405175b3, State: Running, Role: LEADER
I20260809 16:13:42.132916 7021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2955c8768961479cb210dd61405175b3" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40765 } }
I20260809 16:13:42.133267 7021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2955c8768961479cb210dd61405175b3. Latest consensus state: current_term: 1 leader_uuid: "2955c8768961479cb210dd61405175b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2955c8768961479cb210dd61405175b3" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40765 } } }
I20260809 16:13:42.133291 7020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2955c8768961479cb210dd61405175b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2955c8768961479cb210dd61405175b3" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 40765 } } }
I20260809 16:13:42.133414 7021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:42.133421 7020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:42.133641 7039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:13:42.134656 7039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:13:42.135761 7018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:42.125645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.16.224.190:40765)
0809 16:13:42.125801 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:42.125809 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:42.125836 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:13:42.126266 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:42.126272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:42.126346 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:42.126466 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:42.127195 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:42.128118 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:42.128867 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:42.129482 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:42.129596 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:42.132090 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:42.132095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:42.132095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:42.132108 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:42.134081 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:42.134503 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:42.134506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:42.134506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:42.134617 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:42.134841 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:42.134842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:42.134842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:42.135147 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:42.135289 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:42.135434 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:13:42.135569 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260809 16:13:42.136159 7039 catalog_manager.cc:1347] Generated new cluster ID: ea8d238b92e2490680e5e79390c6044f
I20260809 16:13:42.136209 7039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:13:42.136727 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:13:42.148687 7039 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:13:42.149216 7039 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:13:42.151371 7039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2955c8768961479cb210dd61405175b3: Generated new TSK 0
I20260809 16:13:42.151580 7039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:13:42.190251 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 6917
2026-08-09T16:13:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1458 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:13:42 dist-test-slave-fgb3 krb5kdc[7065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:13:42 dist-test-slave-fgb3 krb5kdc[7065](info): set up 2 sockets
Aug 09 16:13:42 dist-test-slave-fgb3 krb5kdc[7065](info): commencing operation
krb5kdc: starting...
W20260809 16:13:43.336702 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:13:43 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:13:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:13:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:13:43.455363 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42681
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:43.535835 7084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:43.536031 7084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:43.536051 7084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:43.537125 7084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:43.537158 7084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:43.537171 7084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:43.537182 7084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:43.537199 7084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:43.538663 7084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:43.538880 7084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:43.539058 7084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:43.543798 7092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:43.543813 7097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:43.543809 7093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:43.544075 7084 server_base.cc:1034] running on GCE node
I20260809 16:13:43.544314 7084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:43.544607 7084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:43.545750 7084 hybrid_clock.cc:648] HybridClock initialized: now 1786292023545732 us; error 35 us; skew 500 ppm
Aug 09 16:13:43 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292023, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:43.548098 7084 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:13:43.548511 7084 webserver.cc:460] Webserver started at http://127.16.224.190:43831/ using document root <none> and password file <none>
I20260809 16:13:43.548730 7084 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:43.548776 7084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:43.548890 7084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:43.550007 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "d0932f8cdd874ee8a871812c33d11bb8"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "2dcdf0db99281b3ea093b893e17c7868"
server_key_iv: "ae9d9de3581f8143722a0353af0348da"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.550361 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "d0932f8cdd874ee8a871812c33d11bb8"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "2dcdf0db99281b3ea093b893e17c7868"
server_key_iv: "ae9d9de3581f8143722a0353af0348da"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.551672 7084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:43.552412 7106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.552549 7084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.552639 7084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "d0932f8cdd874ee8a871812c33d11bb8"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "2dcdf0db99281b3ea093b893e17c7868"
server_key_iv: "ae9d9de3581f8143722a0353af0348da"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.552698 7084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:43.581565 7084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:43.583504 7084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:43.583657 7084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:43.587625 7084 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:42681
I20260809 16:13:43.587667 7185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:42681 every 8 connection(s)
I20260809 16:13:43.588021 7084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:13:43.589058 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:43.590164 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 7084
I20260809 16:13:43.590265 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:13:43.590488 17282 external_mini_cluster.cc:1428] Setting key 07e7daf1b30231148ab992b9cb565242
I20260809 16:13:43.591888 7187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Bootstrap starting.
I20260809 16:13:43.592533 7187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:43.592859 7187 log.cc:824] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:43.593609 7187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: No bootstrap required, opened a new log
I20260809 16:13:43.595757 7187 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:43.595911 7187 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:43.595933 7187 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0932f8cdd874ee8a871812c33d11bb8, State: Initialized, Role: FOLLOWER
I20260809 16:13:43.596051 7187 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:43.596127 7187 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:13:43.596149 7187 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:13:43.596195 7187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:43.596725 7187 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:43.596825 7187 leader_election.cc:302] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0932f8cdd874ee8a871812c33d11bb8; no voters:
I20260809 16:13:43.597007 7187 leader_election.cc:288] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:13:43.597044 7197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:13:43.597157 7197 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 LEADER]: Becoming Leader. State: Replica: d0932f8cdd874ee8a871812c33d11bb8, State: Running, Role: LEADER
I20260809 16:13:43.597285 7187 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 16:13:43 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:43.597329 7197 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:43.597707 7197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0932f8cdd874ee8a871812c33d11bb8. Latest consensus state: current_term: 1 leader_uuid: "d0932f8cdd874ee8a871812c33d11bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } } }
I20260809 16:13:43.597831 7197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:43.597983 7198 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0932f8cdd874ee8a871812c33d11bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } } }
I20260809 16:13:43.598078 7198 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:43.598328 7205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:13:43.598811 7205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:13:43.601691 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:43.591004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.16.224.190:42681)
0809 16:13:43.591214 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:43.591224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:43.591259 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:13:43.591764 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:43.591774 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:43.591837 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:43.591994 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:43.592560 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.593861 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.594647 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.595280 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.595421 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:43.598041 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:43.598046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:43.598046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:43.598059 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:43.600161 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:43.600494 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:43.600497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:43.600497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:43.600602 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:43.600819 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:43.600819 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:43.600820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:43.601126 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:43.601281 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:43.601350 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 16:13:43.601491 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260809 16:13:43.602732 7205 catalog_manager.cc:1347] Generated new cluster ID: cd0614eaa1df482b8038f661597a1dd9
I20260809 16:13:43.602778 7205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:13:43.623767 7205 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:13:43.624306 7205 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:13:43.634371 7205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Generated new TSK 0
I20260809 16:13:43.634668 7205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:13:43.668803 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42681
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:43.777136 7223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:43.777328 7223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:43.777348 7223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:43.778448 7223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:43.778482 7223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:43.778506 7223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:13:43.779893 7223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:43.780079 7223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:43.780256 7223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:43.785065 7231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:43.785068 7237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:43.785106 7246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:43.785315 7223 server_base.cc:1034] running on GCE node
I20260809 16:13:43.785512 7223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:43.785733 7223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:43.786863 7223 hybrid_clock.cc:648] HybridClock initialized: now 1786292023786851 us; error 30 us; skew 500 ppm
Aug 09 16:13:43 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292023, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:43.789178 7223 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:13:43.789595 7223 webserver.cc:460] Webserver started at http://127.16.224.129:34247/ using document root <none> and password file <none>
I20260809 16:13:43.789798 7223 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:43.789862 7223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:43.789983 7223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:43.791079 7223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "260cc7107a574c508754d2f2949b646e"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "822429b7ca46d0b674faefa16859db17"
server_key_iv: "42c1943492d20f183b75ac0f132fc8de"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.791419 7223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "260cc7107a574c508754d2f2949b646e"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "822429b7ca46d0b674faefa16859db17"
server_key_iv: "42c1943492d20f183b75ac0f132fc8de"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.792587 7223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:13:43.793226 7254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.793370 7223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:13:43.793463 7223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "260cc7107a574c508754d2f2949b646e"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "822429b7ca46d0b674faefa16859db17"
server_key_iv: "42c1943492d20f183b75ac0f132fc8de"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.793527 7223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:43.819820 7223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:43.821906 7223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:43.822044 7223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:43.822264 7223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:43.822613 7223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:43.822649 7223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.822675 7223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:43.822700 7223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.828271 7223 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:37053
I20260809 16:13:43.828341 7439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:37053 every 8 connection(s)
I20260809 16:13:43.828613 7223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:13:43 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292023, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:43.835242 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 7223
I20260809 16:13:43.835358 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:13:43.835583 17282 external_mini_cluster.cc:1428] Setting key a80e039de06cfa9c5ed0c58b4273f13d
I20260809 16:13:43.837384 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:43.829348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:38471 (local address 127.16.224.190:42681)
0809 16:13:43.829411 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:43.829414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:43.829594 (+ 180us) server_negotiation.cc:407] Connection header received
0809 16:13:43.830101 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:43.830102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:43.830117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:43.830169 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:43.831127 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.831635 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.832596 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.832816 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.832870 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:43.834832 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:43.834834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:43.834834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:43.834841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:43.836109 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:43.836538 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:43.836539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:43.836539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:43.836596 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:43.836830 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:43.836831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:43.836831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:43.836927 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:43.836975 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:43.837251 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 16:13:43.837320 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:13:43.837595 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:43.829290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:38471)
0809 16:13:43.829479 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:43.829517 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:43.830008 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:43.830241 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:43.830246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:43.830841 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:43.831032 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:43.831037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.831717 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.831719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:43.832517 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:43.832519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.832885 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.832886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:43.833013 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:43.833389 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:43.833396 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:43.834730 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:43.836187 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:43.836195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:43.836197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:43.836467 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:43.836654 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:43.836655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:43.836655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:43.836779 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:43.836981 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:43.836987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:43.837137 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 16:13:43.837240 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:13:43.837389 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260809 16:13:43.838079 7441 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42681
I20260809 16:13:43.838176 7441 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:43.838378 7441 heartbeater.cc:511] Master 127.16.224.190:42681 requested a full tablet report, sending...
I20260809 16:13:43.838878 7127 ts_manager.cc:194] Registered new tserver with Master: 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053)
I20260809 16:13:43.839483 7127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:38471
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:13:43.860845 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42681
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:43.941813 7446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:43.942032 7446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:43.942057 7446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:43.943272 7446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:43.943317 7446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:43.943343 7446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:13:43.944748 7446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:43.944944 7446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:43.945137 7446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:43.949967 7453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:43.949992 7446 server_base.cc:1034] running on GCE node
W20260809 16:13:43.949985 7452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:43.949975 7457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:43.950383 7446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:43.950661 7446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:43.951805 7446 hybrid_clock.cc:648] HybridClock initialized: now 1786292023951789 us; error 29 us; skew 500 ppm
Aug 09 16:13:43 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292023, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:43.954196 7446 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:13:43.954659 7446 webserver.cc:460] Webserver started at http://127.16.224.130:34841/ using document root <none> and password file <none>
I20260809 16:13:43.954838 7446 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:43.954878 7446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:43.955015 7446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:43.956094 7446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "dfe2f784120c4b2d968e05cefacb5759"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "da50283ed1d6ba8d570a415287a1aa42"
server_key_iv: "4029ed74f659df347a2116c5c7c297bb"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.956430 7446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "dfe2f784120c4b2d968e05cefacb5759"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "da50283ed1d6ba8d570a415287a1aa42"
server_key_iv: "4029ed74f659df347a2116c5c7c297bb"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.957696 7446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:13:43.958427 7467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.958585 7446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.958683 7446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "dfe2f784120c4b2d968e05cefacb5759"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "da50283ed1d6ba8d570a415287a1aa42"
server_key_iv: "4029ed74f659df347a2116c5c7c297bb"
server_key_version: "encryptionkey@0"
I20260809 16:13:43.958747 7446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:43.970779 7446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:43.972637 7446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:43.972797 7446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:43.973034 7446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:43.973356 7446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:43.973392 7446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.973423 7446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:43.973438 7446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:43.979156 7446 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:35065
I20260809 16:13:43.979271 7662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:35065 every 8 connection(s)
I20260809 16:13:43.979624 7446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:13:43 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292023, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:43.986233 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 7446
I20260809 16:13:43.986367 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:13:43.986588 17282 external_mini_cluster.cc:1428] Setting key f07a0214fbfc90a77d206b78ad8b8068
I20260809 16:13:43.988732 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:43.980346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:38001 (local address 127.16.224.190:42681)
0809 16:13:43.980409 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:43.980411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:43.980618 (+ 207us) server_negotiation.cc:407] Connection header received
0809 16:13:43.981142 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:43.981143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:43.981157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:43.981210 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:43.982183 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.982757 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.983736 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.983947 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.984005 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:43.986028 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:43.986030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:43.986030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:43.986037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:43.987467 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:43.987889 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:43.987889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:43.987890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:43.987937 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:43.988172 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:43.988172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:43.988172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:43.988268 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:43.988311 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:43.988605 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 16:13:43.988668 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:13:43.988965 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:43.980300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:38001)
0809 16:13:43.980492 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:43.980530 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:43.981042 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:43.981283 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:43.981287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:43.981881 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:43.982087 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:43.982092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.982847 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.982848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:43.983654 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:43.983656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:43.984017 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:43.984017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:43.984136 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:43.984585 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:43.984593 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:43.985924 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:43.987540 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:43.987547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:43.987550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:43.987815 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:43.987988 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:43.987989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:43.987989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:43.988117 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:43.988314 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:43.988327 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:43.988481 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 16:13:43.988585 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 16:13:43.988741 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
I20260809 16:13:43.989405 7666 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42681
I20260809 16:13:43.989518 7666 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:43.989707 7666 heartbeater.cc:511] Master 127.16.224.190:42681 requested a full tablet report, sending...
I20260809 16:13:43.990049 7127 ts_manager.cc:194] Registered new tserver with Master: dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065)
I20260809 16:13:43.990489 7127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:38001
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:13:44.011161 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42681
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:44.092123 7673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:44.092329 7673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:44.092351 7673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:44.093489 7673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:44.093528 7673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:44.093552 7673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:13:44.095052 7673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:44.095285 7673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:44.095492 7673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:44.100544 7684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.100556 7683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.100529 7689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:44.100760 7673 server_base.cc:1034] running on GCE node
I20260809 16:13:44.100946 7673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:44.101159 7673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:44.102293 7673 hybrid_clock.cc:648] HybridClock initialized: now 1786292024102281 us; error 27 us; skew 500 ppm
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:44.109395 7673 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:13:44.109829 7673 webserver.cc:460] Webserver started at http://127.16.224.131:44355/ using document root <none> and password file <none>
I20260809 16:13:44.110038 7673 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:44.110086 7673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:44.110214 7673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:13:44.111299 7673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "49a445597382481796edf17cd8e83f51"
format_stamp: "Formatted at 2026-08-09 16:13:44 on dist-test-slave-fgb3"
server_key: "6a4bad2e5845bdd1f0ed8f4185c61cf5"
server_key_iv: "e517eb1c593059d96ece6cd65486a201"
server_key_version: "encryptionkey@0"
I20260809 16:13:44.111627 7673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "49a445597382481796edf17cd8e83f51"
format_stamp: "Formatted at 2026-08-09 16:13:44 on dist-test-slave-fgb3"
server_key: "6a4bad2e5845bdd1f0ed8f4185c61cf5"
server_key_iv: "e517eb1c593059d96ece6cd65486a201"
server_key_version: "encryptionkey@0"
I20260809 16:13:44.112847 7673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:13:44.113495 7696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.113664 7673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.113754 7673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "49a445597382481796edf17cd8e83f51"
format_stamp: "Formatted at 2026-08-09 16:13:44 on dist-test-slave-fgb3"
server_key: "6a4bad2e5845bdd1f0ed8f4185c61cf5"
server_key_iv: "e517eb1c593059d96ece6cd65486a201"
server_key_version: "encryptionkey@0"
I20260809 16:13:44.113819 7673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:44.128397 7673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:44.130489 7673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:44.130626 7673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:44.130832 7673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:44.131127 7673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:44.131160 7673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.131188 7673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:44.131212 7673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.137008 7673 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:34041
I20260809 16:13:44.137060 7873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:34041 every 8 connection(s)
I20260809 16:13:44.137362 7673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:44.146240 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.138011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:48055 (local address 127.16.224.190:42681)
0809 16:13:44.138086 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.138089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.138240 (+ 151us) server_negotiation.cc:407] Connection header received
0809 16:13:44.138771 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.138772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.138783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.138822 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:44.139689 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.140182 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.141284 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.141477 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.141528 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.143835 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:44.143836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:44.143837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:44.143843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:44.145111 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.145509 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.145510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.145510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.145555 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.145730 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.145730 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.145730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.145833 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:44.145877 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.146082 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.146148 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:13:44.146417 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.137969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:48055)
0809 16:13:44.138128 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.138165 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.138673 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.138880 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.138883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.139430 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:44.139621 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.139626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.140308 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.140310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.141205 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.141208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.141562 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.141567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.141697 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.142243 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:44.142273 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:44.143747 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:44.145206 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.145213 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.145214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.145445 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.145598 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.145598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.145599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.145689 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.145874 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:44.145877 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:44.145989 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.146066 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.146181 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260809 16:13:44.146417 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 7673
I20260809 16:13:44.146502 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:13:44.146690 17282 external_mini_cluster.cc:1428] Setting key 40618704726f97fbdac7a56bafec36df
I20260809 16:13:44.147081 7874 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42681
I20260809 16:13:44.147174 7874 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:44.147348 7874 heartbeater.cc:511] Master 127.16.224.190:42681 requested a full tablet report, sending...
I20260809 16:13:44.147696 7127 ts_manager.cc:194] Registered new tserver with Master: 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041)
I20260809 16:13:44.148051 7127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:48055
I20260809 16:13:44.158249 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:13:44.164913 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.159602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.16.224.190:42681)
0809 16:13:44.159664 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.159666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.159749 (+ 83us) server_negotiation.cc:407] Connection header received
0809 16:13:44.159822 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.159823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.159833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.159878 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:44.160405 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.160883 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.161658 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.161847 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.161893 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.162975 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:44.162977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:44.162977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:44.162983 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:44.164067 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.164357 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.164358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.164358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.164399 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.164533 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.164533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.164533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.164657 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:44.164726 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.164800 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.164852 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:13:44.166350 7127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:13:44.167835 7127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:13:44.174245 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.170124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50524 (local address 127.16.224.130:35065)
0809 16:13:44.170435 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.170440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.170450 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:13:44.170491 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.170494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.170509 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.170590 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.170733 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.171744 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.173211 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.173873 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.173938 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.173993 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.174126 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.174167 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":87,"threads_started":1}
I20260809 16:13:44.174275 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.169927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:34041 (local address 127.0.0.1:53646)
0809 16:13:44.170042 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.170083 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.170140 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.171044 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.171045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.171047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.171082 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.171083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.172162 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.172163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.173049 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.173051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.173929 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.173930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.174016 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.174030 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.174103 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.174170 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260809 16:13:44.174533 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.170294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.16.224.131:34041)
0809 16:13:44.170761 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.170765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.170774 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:13:44.170886 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.170888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.170903 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.170971 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.171160 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.172077 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.173120 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.173830 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.173930 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.173993 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.174415 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.174459 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":101,"threads_started":1}
I20260809 16:13:44.174278 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.169930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:35065 (local address 127.0.0.1:50524)
0809 16:13:44.170080 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.170090 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.170141 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.170595 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.170597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.170599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.170651 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.170653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.171833 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.171834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.173137 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.173139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.173946 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.173947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.174016 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.174031 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.174103 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.174170 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":121,"thread_start_us":64,"threads_started":1}
I20260809 16:13:44.175310 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.170131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:37053 (local address 127.0.0.1:41616)
0809 16:13:44.170366 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.170375 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.170433 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.170779 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.170781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.170783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.170825 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.170827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.171760 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.171761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.173131 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.173134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.175081 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.175083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.175145 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.175150 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.175210 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.175250 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":66,"thread_start_us":72,"threads_started":1}
I20260809 16:13:44.175362 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.170380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.16.224.129:37053)
0809 16:13:44.170554 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.170557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.170572 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:13:44.170609 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.170611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.170625 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.170690 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.170904 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.171674 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.174030 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.174839 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.174911 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.174967 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.175237 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.175280 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260809 16:13:44.175230 7785 tablet_service.cc:1467] Processing CreateTablet for tablet 31aff7bfa2d24a0bbda8f48e895cac21 (DEFAULT_TABLE table=test-table [id=d3bdf8591ee3488dbd053cead8df70a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:44.175622 7785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31aff7bfa2d24a0bbda8f48e895cac21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:44.175522 7553 tablet_service.cc:1467] Processing CreateTablet for tablet 31aff7bfa2d24a0bbda8f48e895cac21 (DEFAULT_TABLE table=test-table [id=d3bdf8591ee3488dbd053cead8df70a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:44.175817 7553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31aff7bfa2d24a0bbda8f48e895cac21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:44.176015 7329 tablet_service.cc:1467] Processing CreateTablet for tablet 31aff7bfa2d24a0bbda8f48e895cac21 (DEFAULT_TABLE table=test-table [id=d3bdf8591ee3488dbd053cead8df70a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:44.176318 7329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31aff7bfa2d24a0bbda8f48e895cac21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:44.178169 7906 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Bootstrap starting.
I20260809 16:13:44.178812 7906 tablet_bootstrap.cc:654] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:44.178867 7908 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Bootstrap starting.
I20260809 16:13:44.179366 7906 log.cc:824] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:44.179461 7908 tablet_bootstrap.cc:654] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:44.179492 7909 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Bootstrap starting.
I20260809 16:13:44.179961 7908 log.cc:824] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:44.180066 7909 tablet_bootstrap.cc:654] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:44.180194 7906 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: No bootstrap required, opened a new log
I20260809 16:13:44.180262 7906 ts_tablet_manager.cc:1397] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.180613 7909 log.cc:824] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:44.180835 7908 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: No bootstrap required, opened a new log
I20260809 16:13:44.180954 7908 ts_tablet_manager.cc:1397] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.181239 7909 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: No bootstrap required, opened a new log
I20260809 16:13:44.181299 7909 ts_tablet_manager.cc:1397] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:13:44.182133 7906 raft_consensus.cc:348] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.182305 7906 raft_consensus.cc:374] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:44.182384 7906 raft_consensus.cc:729] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe2f784120c4b2d968e05cefacb5759, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.182530 7906 consensus_queue.cc:260] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.182809 7906 ts_tablet_manager.cc:1428] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 16:13:44.182803 7909 raft_consensus.cc:348] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.182838 7908 raft_consensus.cc:348] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.182945 7666 heartbeater.cc:503] Master 127.16.224.190:42681 was elected leader, sending a full tablet report...
I20260809 16:13:44.182941 7909 raft_consensus.cc:374] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:44.182977 7909 raft_consensus.cc:729] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260cc7107a574c508754d2f2949b646e, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.182981 7908 raft_consensus.cc:374] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:44.183017 7908 raft_consensus.cc:729] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a445597382481796edf17cd8e83f51, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.183120 7909 consensus_queue.cc:260] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.183151 7908 consensus_queue.cc:260] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.183416 7908 ts_tablet_manager.cc:1428] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.183418 7874 heartbeater.cc:503] Master 127.16.224.190:42681 was elected leader, sending a full tablet report...
I20260809 16:13:44.183483 7909 ts_tablet_manager.cc:1428] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.183738 7441 heartbeater.cc:503] Master 127.16.224.190:42681 was elected leader, sending a full tablet report...
I20260809 16:13:44.190356 7922 raft_consensus.cc:482] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:44.190423 7922 raft_consensus.cc:504] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.190666 7922 leader_election.cc:288] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065), 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053)
I20260809 16:13:44.193058 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.190893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:34305 (local address 127.16.224.130:35065)
0809 16:13:44.190943 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.190945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.190952 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:13:44.190971 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.190972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.190981 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.191027 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.191151 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.191595 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.192420 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.192773 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.192831 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.192838 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.192969 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.193001 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:13:44.193058 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.190793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:35065 (local address 127.16.224.131:34305)
0809 16:13:44.190846 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.190857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.190893 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.191037 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.191037 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.191039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.191100 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.191101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.191659 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.191660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.192344 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.192346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.192837 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.192838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.192912 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.192915 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.192959 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.192994 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:13:44.193266 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.190833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:51481 (local address 127.16.224.129:37053)
0809 16:13:44.190868 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.190870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.191062 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:13:44.191115 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.191116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.191126 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.191170 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.191291 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.191722 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.192586 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.192945 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.193004 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.193011 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.193170 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.193210 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:13:44.193344 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31aff7bfa2d24a0bbda8f48e895cac21" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe2f784120c4b2d968e05cefacb5759" is_pre_election: true
I20260809 16:13:44.193461 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.190800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:37053 (local address 127.16.224.131:51481)
0809 16:13:44.190996 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.191006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.191059 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.191176 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.191177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.191178 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.191239 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.191241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.191789 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.191790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.192510 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.192512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.193012 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.193013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.193083 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.193087 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.193344 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.193391 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":39,"thread_start_us":100,"threads_started":1}
I20260809 16:13:44.193473 7581 raft_consensus.cc:2454] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 0.
I20260809 16:13:44.193677 7700 leader_election.cc:302] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759; no voters:
I20260809 16:13:44.193794 7922 raft_consensus.cc:2781] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:13:44.193851 7922 raft_consensus.cc:482] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:13:44.193882 7922 raft_consensus.cc:3037] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:44.193948 7366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31aff7bfa2d24a0bbda8f48e895cac21" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260cc7107a574c508754d2f2949b646e" is_pre_election: true
I20260809 16:13:44.194077 7366 raft_consensus.cc:2454] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 0.
I20260809 16:13:44.194563 7922 raft_consensus.cc:504] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.194675 7922 leader_election.cc:288] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 election: Requested vote from peers dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065), 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053)
I20260809 16:13:44.194810 7581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31aff7bfa2d24a0bbda8f48e895cac21" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe2f784120c4b2d968e05cefacb5759"
I20260809 16:13:44.194865 7581 raft_consensus.cc:3037] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:44.195487 7581 raft_consensus.cc:2454] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 1.
I20260809 16:13:44.195648 7700 leader_election.cc:302] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759; no voters:
I20260809 16:13:44.195673 7366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31aff7bfa2d24a0bbda8f48e895cac21" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260cc7107a574c508754d2f2949b646e"
I20260809 16:13:44.195746 7922 raft_consensus.cc:2781] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:13:44.195741 7366 raft_consensus.cc:3037] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:44.195902 7922 raft_consensus.cc:686] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 1 LEADER]: Becoming Leader. State: Replica: 49a445597382481796edf17cd8e83f51, State: Running, Role: LEADER
I20260809 16:13:44.195999 7922 consensus_queue.cc:237] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.196499 7366 raft_consensus.cc:2454] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 1.
I20260809 16:13:44.196777 7127 catalog_manager.cc:5495] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49a445597382481796edf17cd8e83f51 (127.16.224.131). New cstate: current_term: 1 leader_uuid: "49a445597382481796edf17cd8e83f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } health_report { overall_health: UNKNOWN } } }
I20260809 16:13:44.203506 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.201649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.16.224.131:34041)
0809 16:13:44.201686 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.201688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.201694 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:13:44.201717 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.201718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.201731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.201777 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:13:44.201858 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.202270 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.202991 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.203160 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.203202 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.203287 (+ 85us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:13:44.203362 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:13:44.203410 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.203421 (+ 11us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.203433 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:13:44.204892 7366 raft_consensus.cc:1261] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Refusing update from remote peer 49a445597382481796edf17cd8e83f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:44.204895 7581 raft_consensus.cc:1261] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Refusing update from remote peer 49a445597382481796edf17cd8e83f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:44.205192 7926 consensus_queue.cc:1035] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:44.205314 7922 consensus_queue.cc:1035] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:44.207911 7935 mvcc.cc:205] Tried to move back new op lower bound from 7316652131141414912 to 7316652131107446784. Current Snapshot: MvccSnapshot[applied={T|T < 7316652131141414912}]
I20260809 16:13:44.208129 7933 mvcc.cc:205] Tried to move back new op lower bound from 7316652131141414912 to 7316652131107446784. Current Snapshot: MvccSnapshot[applied={T|T < 7316652131141414912}]
I20260809 16:13:44.209587 7127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:13:44.209671 7127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:13:44.210366 7127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Sending DeleteTablet for 3 replicas of tablet 31aff7bfa2d24a0bbda8f48e895cac21
I20260809 16:13:44.210652 7785 tablet_service.cc:1514] Processing DeleteTablet for tablet 31aff7bfa2d24a0bbda8f48e895cac21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:44 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:53646
I20260809 16:13:44.210718 7329 tablet_service.cc:1514] Processing DeleteTablet for tablet 31aff7bfa2d24a0bbda8f48e895cac21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:44 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:41616
I20260809 16:13:44.210736 7553 tablet_service.cc:1514] Processing DeleteTablet for tablet 31aff7bfa2d24a0bbda8f48e895cac21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:44 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:50524
I20260809 16:13:44.210999 7949 tablet_replica.cc:330] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: stopping tablet replica
I20260809 16:13:44.211002 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 7223
I20260809 16:13:44.220461 7949 raft_consensus.cc:2229] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:13:44.220597 7949 raft_consensus.cc:2258] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:44.221017 7949 ts_tablet_manager.cc:1905] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:13:44.222276 7949 ts_tablet_manager.cc:1918] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:13:44.222364 7949 log.cc:1196] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/31aff7bfa2d24a0bbda8f48e895cac21
I20260809 16:13:44.222613 7949 ts_tablet_manager.cc:1939] T 31aff7bfa2d24a0bbda8f48e895cac21 P dfe2f784120c4b2d968e05cefacb5759: Deleting consensus metadata
I20260809 16:13:44.222991 7110 catalog_manager.cc:4841] TS dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065): tablet 31aff7bfa2d24a0bbda8f48e895cac21 (table test-table [id=d3bdf8591ee3488dbd053cead8df70a3]) successfully deleted
I20260809 16:13:44.226456 7948 tablet_replica.cc:330] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: stopping tablet replica
I20260809 16:13:44.226559 7948 raft_consensus.cc:2229] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:13:44.226689 7948 raft_consensus.cc:2258] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:44.227066 7948 ts_tablet_manager.cc:1905] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:13:44.228253 7948 ts_tablet_manager.cc:1918] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:13:44.228314 7948 log.cc:1196] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/31aff7bfa2d24a0bbda8f48e895cac21
I20260809 16:13:44.228531 7948 ts_tablet_manager.cc:1939] T 31aff7bfa2d24a0bbda8f48e895cac21 P 49a445597382481796edf17cd8e83f51: Deleting consensus metadata
I20260809 16:13:44.228876 7107 catalog_manager.cc:4841] TS 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041): tablet 31aff7bfa2d24a0bbda8f48e895cac21 (table test-table [id=d3bdf8591ee3488dbd053cead8df70a3]) successfully deleted
W20260809 16:13:44.229468 7107 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:37053 (error 108)
I20260809 16:13:44.229744 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 7446
I20260809 16:13:44.231345 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.229827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:37053 (local address 127.0.0.1:41622)
0809 16:13:44.231278 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.231320 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:37053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1429}
W20260809 16:13:44.231516 7107 catalog_manager.cc:4584] TS 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31aff7bfa2d24a0bbda8f48e895cac21: Network error: Client connection negotiation failed: client connection to 127.16.224.129:37053: connect: Connection refused (error 111)
I20260809 16:13:44.238143 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 7673
I20260809 16:13:44.248641 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 7084
I20260809 16:13:44.257982 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42681
--webserver_interface=127.16.224.190
--webserver_port=43831
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:44.338641 7956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:44.338835 7956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:44.338855 7956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:44.339973 7956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:44.340014 7956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:44.340027 7956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:44.340050 7956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:44.340075 7956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:44.341583 7956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=43831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:44.341815 7956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:44.342024 7956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:44.351779 7967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.351787 7966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:44.351909 7956 server_base.cc:1034] running on GCE node
W20260809 16:13:44.351845 7969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:44.352252 7956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:44.352576 7956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:44.353724 7956 hybrid_clock.cc:648] HybridClock initialized: now 1786292024353703 us; error 35 us; skew 500 ppm
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:44.356149 7956 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:13:44.356545 7956 webserver.cc:460] Webserver started at http://127.16.224.190:43831/ using document root <none> and password file <none>
I20260809 16:13:44.356745 7956 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:44.356794 7956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:44.358125 7956 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:13:44.358824 7981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.358978 7956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.359067 7956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "d0932f8cdd874ee8a871812c33d11bb8"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "2dcdf0db99281b3ea093b893e17c7868"
server_key_iv: "ae9d9de3581f8143722a0353af0348da"
server_key_version: "encryptionkey@0"
I20260809 16:13:44.359455 7956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:44.383530 7956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:44.385761 7956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:44.385879 7956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:44.389720 7956 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:42681
I20260809 16:13:44.389770 8060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:42681 every 8 connection(s)
I20260809 16:13:44.390143 7956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:13:44.391819 8061 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:13:44.392511 8061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Bootstrap starting.
I20260809 16:13:44.393023 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 7956
I20260809 16:13:44.393289 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:37053
--local_ip_for_outbound_sockets=127.16.224.129
--tserver_master_addrs=127.16.224.190:42681
--webserver_port=34247
--webserver_interface=127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:44.401153 8061 log.cc:824] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:44.403134 8061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:13:44.403405 8061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Bootstrap complete.
I20260809 16:13:44.405602 8061 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:44.405827 8061 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0932f8cdd874ee8a871812c33d11bb8, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.405985 8061 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:44.406061 8061 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:13:44.406117 8061 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:13:44.406165 8061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:13:44.407213 8061 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:44.407330 8061 leader_election.cc:302] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0932f8cdd874ee8a871812c33d11bb8; no voters:
I20260809 16:13:44.407493 8061 leader_election.cc:288] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:13:44.407528 8068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:13:44.407683 8068 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 LEADER]: Becoming Leader. State: Replica: d0932f8cdd874ee8a871812c33d11bb8, State: Running, Role: LEADER
I20260809 16:13:44.407764 8061 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:44.407820 8068 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:44.408136 8069 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0932f8cdd874ee8a871812c33d11bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } } }
I20260809 16:13:44.408128 8070 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0932f8cdd874ee8a871812c33d11bb8. Latest consensus state: current_term: 2 leader_uuid: "d0932f8cdd874ee8a871812c33d11bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } } }
I20260809 16:13:44.408252 8069 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:44.408269 8070 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:44.408461 8075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:13:44.409397 8075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:13:44.409518 8075 catalog_manager.cc:1259] Loaded cluster ID: cd0614eaa1df482b8038f661597a1dd9
I20260809 16:13:44.409551 8075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:13:44.409878 8075 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 16:13:44.414466 8075 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 16:13:44.416795 8075 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:13:44.417037 8075 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Loaded TSK: 0
I20260809 16:13:44.417254 8075 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:13:44.477573 8065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:44.477782 8065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:44.477818 8065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:44.479032 8065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:44.479075 8065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:44.479110 8065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:13:44.480589 8065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:37053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=34247
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:44.480868 8065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:44.481089 8065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:44.486948 8097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.486958 8099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.487002 8101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:44.487228 8065 server_base.cc:1034] running on GCE node
I20260809 16:13:44.487406 8065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:44.487625 8065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:44.488782 8065 hybrid_clock.cc:648] HybridClock initialized: now 1786292024488767 us; error 31 us; skew 500 ppm
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:44.491360 8065 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:13:44.491820 8065 webserver.cc:460] Webserver started at http://127.16.224.129:34247/ using document root <none> and password file <none>
I20260809 16:13:44.492020 8065 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:44.492085 8065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:44.493508 8065 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:44.494200 8111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.494391 8065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:44.494490 8065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "260cc7107a574c508754d2f2949b646e"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "822429b7ca46d0b674faefa16859db17"
server_key_iv: "42c1943492d20f183b75ac0f132fc8de"
server_key_version: "encryptionkey@0"
I20260809 16:13:44.494846 8065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:44.519743 8065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:44.521788 8065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:44.521934 8065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:44.522177 8065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:44.522687 8123 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:13:44.524142 8065 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:13:44.524192 8065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:13:44.524217 8065 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:13:44.524873 8065 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:13:44.524919 8065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:13:44.524993 8123 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Bootstrap starting.
I20260809 16:13:44.532342 8065 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:37053
I20260809 16:13:44.532867 8065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:13:44.534471 8271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:37053 every 8 connection(s)
I20260809 16:13:44.535563 8123 log.cc:824] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:44.538087 8123 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:13:44.538429 8123 tablet_bootstrap.cc:492] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Bootstrap complete.
I20260809 16:13:44.539201 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 8065
I20260809 16:13:44.539423 8123 ts_tablet_manager.cc:1397] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260809 16:13:44.539527 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:35065
--local_ip_for_outbound_sockets=127.16.224.130
--tserver_master_addrs=127.16.224.190:42681
--webserver_port=34841
--webserver_interface=127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:44.540618 8123 raft_consensus.cc:348] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.540902 8123 raft_consensus.cc:729] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260cc7107a574c508754d2f2949b646e, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.541067 8123 consensus_queue.cc:260] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.541335 8123 ts_tablet_manager.cc:1428] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:44.545612 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.533666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:50485 (local address 127.16.224.190:42681)
0809 16:13:44.533856 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.533866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.535978 (+ 2112us) server_negotiation.cc:407] Connection header received
0809 16:13:44.536584 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.536589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.536633 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.536750 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:44.537370 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.538281 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.539526 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.540063 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.540170 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.542542 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:44.542546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:44.542546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:44.542556 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:44.544063 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.544435 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.544438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.544438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.544521 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.544740 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.544741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.544742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.544955 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:44.545027 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.545301 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.545430 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260809 16:13:44.545611 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.533546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:50485)
0809 16:13:44.535803 (+ 2257us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.535847 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.536369 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.536798 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.536802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.537073 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:44.537255 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.537260 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.538399 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.538401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.539401 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.539405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.540183 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.540185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.540331 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.540912 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:44.540931 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:44.542431 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:44.544141 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.544147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.544150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.544354 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.544582 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.544583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.544583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.544685 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.545020 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:44.545026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:44.545155 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.545242 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.545414 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2214,"thread_start_us":92,"threads_started":1}
I20260809 16:13:44.546221 8272 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42681
I20260809 16:13:44.546334 8272 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:44.546504 8272 heartbeater.cc:511] Master 127.16.224.190:42681 requested a full tablet report, sending...
I20260809 16:13:44.547158 8003 ts_manager.cc:194] Registered new tserver with Master: 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053)
I20260809 16:13:44.548154 8003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:50485
I20260809 16:13:44.548941 8272 heartbeater.cc:503] Master 127.16.224.190:42681 was elected leader, sending a full tablet report...
I20260809 16:13:44.553603 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.547780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.16.224.129:37053)
0809 16:13:44.547927 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.547931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.550530 (+ 2599us) server_negotiation.cc:407] Connection header received
0809 16:13:44.550627 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.550629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.550642 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.550703 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.550836 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.551463 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.552875 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.553292 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.553345 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.553377 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.553533 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.553558 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260809 16:13:44.553651 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.547701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:37053 (local address 127.0.0.1:41630)
0809 16:13:44.550431 (+ 2730us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.550470 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.550544 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.550720 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.550722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.550724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.550780 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.550781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.551522 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.551522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.552804 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.552807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.553357 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.553358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.553443 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.553458 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.553521 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.553574 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2691,"thread_start_us":83,"threads_started":1}
I20260809 16:13:44.554071 8188 tablet_service.cc:1514] Processing DeleteTablet for tablet 31aff7bfa2d24a0bbda8f48e895cac21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:44 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:41630
I20260809 16:13:44.554569 8284 tablet_replica.cc:330] stopping tablet replica
I20260809 16:13:44.554674 8284 raft_consensus.cc:2229] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:13:44.554737 8284 raft_consensus.cc:2258] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:44.555029 8284 ts_tablet_manager.cc:1905] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:13:44.557179 8284 ts_tablet_manager.cc:1918] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:13:44.557255 8284 log.cc:1196] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/31aff7bfa2d24a0bbda8f48e895cac21
I20260809 16:13:44.557482 8284 ts_tablet_manager.cc:1939] T 31aff7bfa2d24a0bbda8f48e895cac21 P 260cc7107a574c508754d2f2949b646e: Deleting consensus metadata
I20260809 16:13:44.558118 7982 catalog_manager.cc:4841] TS 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053): tablet 31aff7bfa2d24a0bbda8f48e895cac21 (table test-table [id=d3bdf8591ee3488dbd053cead8df70a3]) successfully deleted
W20260809 16:13:44.624579 8279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:44.624773 8279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:44.624794 8279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:44.625885 8279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:44.625921 8279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:44.625945 8279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:13:44.627377 8279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:35065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=34841
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:44.627575 8279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:44.627749 8279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:44.632557 8297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.632566 8298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:44.632572 8279 server_base.cc:1034] running on GCE node
W20260809 16:13:44.632557 8300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:44.632948 8279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:44.633144 8279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:44.634274 8279 hybrid_clock.cc:648] HybridClock initialized: now 1786292024634261 us; error 29 us; skew 500 ppm
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:44.636693 8279 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:13:44.637123 8279 webserver.cc:460] Webserver started at http://127.16.224.130:34841/ using document root <none> and password file <none>
I20260809 16:13:44.637367 8279 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:44.637428 8279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:44.638928 8279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:13:44.639503 8309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.639676 8279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.639775 8279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "dfe2f784120c4b2d968e05cefacb5759"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "da50283ed1d6ba8d570a415287a1aa42"
server_key_iv: "4029ed74f659df347a2116c5c7c297bb"
server_key_version: "encryptionkey@0"
I20260809 16:13:44.640108 8279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:44.647565 8279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:44.649130 8279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:44.649256 8279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:44.649446 8279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:44.649724 8279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:44.649755 8279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.649781 8279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:44.649816 8279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.655247 8279 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:35065
I20260809 16:13:44.655311 8489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:35065 every 8 connection(s)
I20260809 16:13:44.655602 8279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:44.664644 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.656266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:60745 (local address 127.16.224.190:42681)
0809 16:13:44.656321 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.656323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.656522 (+ 199us) server_negotiation.cc:407] Connection header received
0809 16:13:44.657028 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.657030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.657046 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.657098 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:44.658070 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.658704 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.659877 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.660089 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.660145 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.662127 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:44.662128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:44.662129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:44.662137 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:44.663433 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.663783 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.663785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.663785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.663837 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.664050 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.664051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.664051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.664157 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:44.664214 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.664464 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.664554 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:13:44.664772 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.656260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:60745)
0809 16:13:44.656412 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.656447 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.656929 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.657170 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.657174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.657756 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:44.657949 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.657954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.658890 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.658891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.659792 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.659795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.660162 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.660163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.660272 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.660741 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:44.660751 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:44.662038 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:44.663507 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.663514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.663516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.663719 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.663890 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.663891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.663892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.663996 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.664212 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:44.664218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:44.664345 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.664430 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.664572 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260809 16:13:44.665207 8490 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42681
I20260809 16:13:44.665268 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 8279
I20260809 16:13:44.665306 8490 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:44.665484 8490 heartbeater.cc:511] Master 127.16.224.190:42681 requested a full tablet report, sending...
I20260809 16:13:44.665589 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:34041
--local_ip_for_outbound_sockets=127.16.224.131
--tserver_master_addrs=127.16.224.190:42681
--webserver_port=44355
--webserver_interface=127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:44.665856 8003 ts_manager.cc:194] Registered new tserver with Master: dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065)
I20260809 16:13:44.666213 8003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:60745
W20260809 16:13:44.746548 8497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:44.746748 8497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:44.746771 8497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:44.747905 8497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:44.747947 8497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:44.747972 8497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:13:44.749392 8497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:34041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=44355
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:44.749634 8497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:44.749851 8497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:44.754887 8505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.754966 8507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:44.754920 8504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:44.755251 8497 server_base.cc:1034] running on GCE node
I20260809 16:13:44.755463 8497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:44.755697 8497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:44.756868 8497 hybrid_clock.cc:648] HybridClock initialized: now 1786292024756847 us; error 35 us; skew 500 ppm
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:44.759457 8497 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:13:44.759896 8497 webserver.cc:460] Webserver started at http://127.16.224.131:44355/ using document root <none> and password file <none>
I20260809 16:13:44.760120 8497 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:44.760181 8497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:44.761526 8497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:44.762099 8521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.762254 8497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:13:44.762359 8497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "49a445597382481796edf17cd8e83f51"
format_stamp: "Formatted at 2026-08-09 16:13:44 on dist-test-slave-fgb3"
server_key: "6a4bad2e5845bdd1f0ed8f4185c61cf5"
server_key_iv: "e517eb1c593059d96ece6cd65486a201"
server_key_version: "encryptionkey@0"
I20260809 16:13:44.762717 8497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:44.772964 8497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:44.774888 8497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:44.775012 8497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:44.775203 8497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:44.775484 8497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:44.775519 8497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.775547 8497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:44.775569 8497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.783006 8497 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:34041
I20260809 16:13:44.783066 8690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:34041 every 8 connection(s)
I20260809 16:13:44.783376 8497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:13:44 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292024, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:13:44.790962 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 8497
I20260809 16:13:44.791903 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.784022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:39745 (local address 127.16.224.190:42681)
0809 16:13:44.784105 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.784107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.784279 (+ 172us) server_negotiation.cc:407] Connection header received
0809 16:13:44.784764 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.784765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.784776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.784824 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:44.785708 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.786192 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.787166 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.787359 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.787406 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.789277 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:44.789279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:44.789279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:44.789287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:44.790541 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.790872 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.790873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.790873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.790943 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.791158 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.791159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.791159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.791277 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:44.791342 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.791724 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.791813 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:13:44.792049 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.783993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:39745)
0809 16:13:44.784149 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.784185 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.784673 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.784907 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.784910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.785455 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:44.785639 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.785644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.786253 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.786254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.787102 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.787104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.787416 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.787416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.787511 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.787937 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:44.787945 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:44.789194 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:44.790606 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.790614 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.790616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.790810 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.790999 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:44.791000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:44.791001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:44.791099 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:44.791460 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:44.791468 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:44.791586 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.791678 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.791829 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260809 16:13:44.792496 8691 heartbeater.cc:347] Connected to a master server at 127.16.224.190:42681
I20260809 16:13:44.792589 8691 heartbeater.cc:464] Registering TS with master...
I20260809 16:13:44.792761 8691 heartbeater.cc:511] Master 127.16.224.190:42681 requested a full tablet report, sending...
I20260809 16:13:44.793151 8003 ts_manager.cc:194] Registered new tserver with Master: 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041)
I20260809 16:13:44.793502 8003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:39745
I20260809 16:13:44.796829 8699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.791366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39136 (local address 127.16.224.190:42681)
0809 16:13:44.791473 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.791474 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.791510 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:13:44.791577 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.791578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.791588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.791640 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:44.792199 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.792821 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.793622 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.793883 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.793950 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.794992 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:44.794993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:44.794993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:44.794999 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:44.796044 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.796312 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.796313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.796313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.796356 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:44.796486 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:44.796487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:44.796487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:44.796569 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:44.796683 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.796709 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.796766 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260809 16:13:44.797508 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:13:44.800122 8699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.797828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39144 (local address 127.16.224.190:42681)
0809 16:13:44.797863 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.797865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.797960 (+ 95us) server_negotiation.cc:407] Connection header received
0809 16:13:44.798046 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.798047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.798084 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.798140 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:13:44.798249 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.798738 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.799436 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.799642 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.799696 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.799838 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:13:44.799937 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:13:44.800002 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.800038 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.800059 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:13:44.800376 8003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:13:44.801926 8003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:13:44.804020 8188 tablet_service.cc:1467] Processing CreateTablet for tablet 7a3003e679da40e2a0a68359fbe926a4 (DEFAULT_TABLE table=test-table [id=a9e38315ad524b91b64680d5ae84af7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:44.804234 8188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a3003e679da40e2a0a68359fbe926a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:44.805807 8123 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Bootstrap starting.
I20260809 16:13:44.806093 8123 tablet_bootstrap.cc:654] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:44.807107 8123 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: No bootstrap required, opened a new log
I20260809 16:13:44.807154 8123 ts_tablet_manager.cc:1397] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:13:44.807245 8123 raft_consensus.cc:348] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.807298 8123 raft_consensus.cc:374] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:44.807331 8123 raft_consensus.cc:729] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260cc7107a574c508754d2f2949b646e, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.807365 8123 consensus_queue.cc:260] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.807452 8123 ts_tablet_manager.cc:1428] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:44.807616 8701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.803718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:35065 (local address 127.0.0.1:50530)
0809 16:13:44.803893 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.803903 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.803947 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.804265 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.804266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.804268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.804306 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.804307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.805246 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.805247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.806224 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.806226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.807095 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.807096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.807253 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.807258 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.807291 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.807546 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260809 16:13:44.807703 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.803671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:34041 (local address 127.0.0.1:53672)
0809 16:13:44.803715 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.803726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.803764 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.804343 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.804345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.804346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.804387 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.804388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.805588 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.805589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.806590 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.806593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.807495 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.807496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.807553 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.807556 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.807584 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.807620 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":12}
I20260809 16:13:44.808099 8702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.803766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53672 (local address 127.16.224.131:34041)
0809 16:13:44.804157 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.804162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.804172 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:13:44.804207 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.804208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.804223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.804459 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.804524 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.805483 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.806712 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.807407 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.807749 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.807818 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.807977 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.808019 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":88,"threads_started":1}
I20260809 16:13:44.808151 8706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.803960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50530 (local address 127.16.224.130:35065)
0809 16:13:44.804111 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.804115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.804126 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:13:44.804167 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.804170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.804184 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.804256 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.804356 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.805161 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.806308 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.806999 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.807906 (+ 907us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.808013 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.808037 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.808076 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260809 16:13:44.808892 8595 tablet_service.cc:1467] Processing CreateTablet for tablet 7a3003e679da40e2a0a68359fbe926a4 (DEFAULT_TABLE table=test-table [id=a9e38315ad524b91b64680d5ae84af7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:44.808892 8391 tablet_service.cc:1467] Processing CreateTablet for tablet 7a3003e679da40e2a0a68359fbe926a4 (DEFAULT_TABLE table=test-table [id=a9e38315ad524b91b64680d5ae84af7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:13:44.809250 8391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a3003e679da40e2a0a68359fbe926a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:44.809250 8595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a3003e679da40e2a0a68359fbe926a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:13:44.812737 8710 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Bootstrap starting.
I20260809 16:13:44.812737 8708 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Bootstrap starting.
I20260809 16:13:44.813388 8708 tablet_bootstrap.cc:654] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:44.813398 8710 tablet_bootstrap.cc:654] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Neither blocks nor log segments found. Creating new log.
I20260809 16:13:44.813951 8710 log.cc:824] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:44.813970 8708 log.cc:824] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:44.814709 8710 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: No bootstrap required, opened a new log
I20260809 16:13:44.814772 8710 ts_tablet_manager.cc:1397] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.814849 8708 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: No bootstrap required, opened a new log
I20260809 16:13:44.814913 8708 ts_tablet_manager.cc:1397] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.816236 8710 raft_consensus.cc:348] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.816251 8708 raft_consensus.cc:348] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.816365 8708 raft_consensus.cc:374] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:44.816383 8708 raft_consensus.cc:729] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe2f784120c4b2d968e05cefacb5759, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.816366 8710 raft_consensus.cc:374] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:13:44.816407 8710 raft_consensus.cc:729] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a445597382481796edf17cd8e83f51, State: Initialized, Role: FOLLOWER
I20260809 16:13:44.816496 8708 consensus_queue.cc:260] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.816560 8710 consensus_queue.cc:260] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.816784 8708 ts_tablet_manager.cc:1428] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.816794 8490 heartbeater.cc:503] Master 127.16.224.190:42681 was elected leader, sending a full tablet report...
I20260809 16:13:44.816838 8710 ts_tablet_manager.cc:1428] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:13:44.816856 8691 heartbeater.cc:503] Master 127.16.224.190:42681 was elected leader, sending a full tablet report...
I20260809 16:13:44.903033 8718 raft_consensus.cc:482] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:44.903157 8718 raft_consensus.cc:504] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.903556 8718 leader_election.cc:288] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065), 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053)
W20260809 16:13:44.906126 8494 tablet.cc:2369] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:13:44.906373 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.903789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:50217 (local address 127.16.224.129:37053)
0809 16:13:44.903858 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.903859 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.903868 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:13:44.903899 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.903900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.903910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.903985 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.904150 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.904669 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.905663 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.906039 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.906098 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.906109 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.906240 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.906278 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:13:44.906373 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.903713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:37053 (local address 127.16.224.131:50217)
0809 16:13:44.903758 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.903770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.903809 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.904008 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.904009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.904010 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.904083 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.904085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.904740 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.904741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.905576 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.905579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.906111 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.906112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.906181 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.906184 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.906228 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.906268 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":1}
I20260809 16:13:44.906682 8706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.903806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:47383 (local address 127.16.224.130:35065)
0809 16:13:44.903848 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.903850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.904044 (+ 194us) server_negotiation.cc:407] Connection header received
0809 16:13:44.904079 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.904080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.904090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.904143 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:13:44.904285 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.904762 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.905646 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.906034 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.906102 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.906110 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.906582 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.906627 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:13:44.906685 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.903715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:35065 (local address 127.16.224.131:47383)
0809 16:13:44.903968 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.903979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:44.904038 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:44.904148 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:44.904149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:44.904150 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:13:44.904220 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.904221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.904833 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.904834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.905569 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:44.905571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.906426 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.906427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:44.906513 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.906518 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:13:44.906574 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:13:44.906606 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":48,"thread_start_us":135,"threads_started":1}
I20260809 16:13:44.906863 8212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3003e679da40e2a0a68359fbe926a4" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260cc7107a574c508754d2f2949b646e" is_pre_election: true
I20260809 16:13:44.907032 8212 raft_consensus.cc:2454] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 0.
I20260809 16:13:44.907056 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3003e679da40e2a0a68359fbe926a4" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe2f784120c4b2d968e05cefacb5759" is_pre_election: true
I20260809 16:13:44.907204 8421 raft_consensus.cc:2454] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 0.
I20260809 16:13:44.907258 8523 leader_election.cc:302] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 260cc7107a574c508754d2f2949b646e, 49a445597382481796edf17cd8e83f51; no voters:
I20260809 16:13:44.907375 8718 raft_consensus.cc:2781] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:13:44.907440 8718 raft_consensus.cc:482] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:13:44.907462 8718 raft_consensus.cc:3037] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:44.908245 8718 raft_consensus.cc:504] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.908437 8718 leader_election.cc:288] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 election: Requested vote from peers dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065), 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053)
I20260809 16:13:44.908519 8212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3003e679da40e2a0a68359fbe926a4" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260cc7107a574c508754d2f2949b646e"
I20260809 16:13:44.908535 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3003e679da40e2a0a68359fbe926a4" candidate_uuid: "49a445597382481796edf17cd8e83f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe2f784120c4b2d968e05cefacb5759"
I20260809 16:13:44.908593 8421 raft_consensus.cc:3037] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:44.908595 8212 raft_consensus.cc:3037] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:13:44.909323 8421 raft_consensus.cc:2454] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 1.
I20260809 16:13:44.909346 8212 raft_consensus.cc:2454] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a445597382481796edf17cd8e83f51 in term 1.
I20260809 16:13:44.909499 8527 leader_election.cc:302] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759; no voters:
I20260809 16:13:44.909596 8718 raft_consensus.cc:2781] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:13:44.909756 8718 raft_consensus.cc:686] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 1 LEADER]: Becoming Leader. State: Replica: 49a445597382481796edf17cd8e83f51, State: Running, Role: LEADER
I20260809 16:13:44.909844 8718 consensus_queue.cc:237] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:44.910679 8003 catalog_manager.cc:5495] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49a445597382481796edf17cd8e83f51 (127.16.224.131). New cstate: current_term: 1 leader_uuid: "49a445597382481796edf17cd8e83f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } health_report { overall_health: UNKNOWN } } }
I20260809 16:13:44.921041 8702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.919111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53682 (local address 127.16.224.131:34041)
0809 16:13:44.919177 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:44.919179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:44.919189 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:13:44.919231 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:44.919231 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:44.919247 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:44.919296 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:13:44.919386 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.919826 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.920462 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:44.920638 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:44.920678 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:44.920773 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:13:44.920865 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:13:44.920924 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:44.920952 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:13:44.920968 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:13:44.922830 8421 raft_consensus.cc:1261] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Refusing update from remote peer 49a445597382481796edf17cd8e83f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:44.922873 8212 raft_consensus.cc:1261] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Refusing update from remote peer 49a445597382481796edf17cd8e83f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:13:44.923182 8721 consensus_queue.cc:1035] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:44.923317 8718 consensus_queue.cc:1035] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:13:44.926020 8726 mvcc.cc:205] Tried to move back new op lower bound from 7316652134081179648 to 7316652134031392768. Current Snapshot: MvccSnapshot[applied={T|T < 7316652134081179648}]
I20260809 16:13:44.927693 8002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:13:44.927781 8002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:13:44.928478 8002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Sending DeleteTablet for 3 replicas of tablet 7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:44.928759 8188 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a3003e679da40e2a0a68359fbe926a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:44 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:41630
I20260809 16:13:44.928750 8595 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a3003e679da40e2a0a68359fbe926a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:44 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:53672
I20260809 16:13:44.928792 8391 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a3003e679da40e2a0a68359fbe926a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:13:44 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:50530
I20260809 16:13:44.928853 8284 tablet_replica.cc:330] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: stopping tablet replica
I20260809 16:13:44.928905 8284 raft_consensus.cc:2229] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:13:44.928946 8284 raft_consensus.cc:2258] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:44.928943 8737 tablet_replica.cc:330] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: stopping tablet replica
I20260809 16:13:44.929023 8737 raft_consensus.cc:2229] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:13:44.929136 8737 raft_consensus.cc:2258] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:44.929175 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 8065
I20260809 16:13:44.935956 8737 ts_tablet_manager.cc:1905] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:13:44.937561 8737 ts_tablet_manager.cc:1918] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:13:44.937644 8737 log.cc:1196] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:44.937932 8737 ts_tablet_manager.cc:1939] T 7a3003e679da40e2a0a68359fbe926a4 P 49a445597382481796edf17cd8e83f51: Deleting consensus metadata
I20260809 16:13:44.938411 7982 catalog_manager.cc:4841] TS 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041): tablet 7a3003e679da40e2a0a68359fbe926a4 (table test-table [id=a9e38315ad524b91b64680d5ae84af7e]) successfully deleted
I20260809 16:13:44.942946 8740 tablet_replica.cc:330] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: stopping tablet replica
I20260809 16:13:44.943104 8740 raft_consensus.cc:2229] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:13:44.943224 8740 raft_consensus.cc:2258] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:13:44.943717 8740 ts_tablet_manager.cc:1905] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:13:44.945328 8740 ts_tablet_manager.cc:1918] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:13:44.945405 8740 log.cc:1196] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:44.945667 8740 ts_tablet_manager.cc:1939] T 7a3003e679da40e2a0a68359fbe926a4 P dfe2f784120c4b2d968e05cefacb5759: Deleting consensus metadata
I20260809 16:13:44.946115 7986 catalog_manager.cc:4841] TS dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065): tablet 7a3003e679da40e2a0a68359fbe926a4 (table test-table [id=a9e38315ad524b91b64680d5ae84af7e]) successfully deleted
W20260809 16:13:44.947350 7982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:37053 (error 108)
I20260809 16:13:44.947499 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 8279
I20260809 16:13:44.951628 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:44.951502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:37053 (local address 127.0.0.1:41646)
0809 16:13:44.951557 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:44.951593 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:37053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 16:13:44.951745 7982 catalog_manager.cc:4584] TS 260cc7107a574c508754d2f2949b646e (127.16.224.129:37053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a3003e679da40e2a0a68359fbe926a4: Network error: Client connection negotiation failed: client connection to 127.16.224.129:37053: connect: Connection refused (error 111)
I20260809 16:13:44.957829 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 8497
I20260809 16:13:44.972601 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 7956
I20260809 16:13:44.981530 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42681
--webserver_interface=127.16.224.190
--webserver_port=43831
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:45.062219 8742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:45.062453 8742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:45.062484 8742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:45.063608 8742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:13:45.063647 8742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:45.063665 8742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:45.063681 8742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:13:45.063697 8742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:13:45.065150 8742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=43831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:45.065408 8742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:45.065616 8742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:45.070135 8758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:45.070217 8742 server_base.cc:1034] running on GCE node
W20260809 16:13:45.070109 8754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:45.070117 8753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:45.070551 8742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:45.070819 8742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:45.071964 8742 hybrid_clock.cc:648] HybridClock initialized: now 1786292025071950 us; error 29 us; skew 500 ppm
Aug 09 16:13:45 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292025, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:45.074348 8742 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:13:45.074720 8742 webserver.cc:460] Webserver started at http://127.16.224.190:43831/ using document root <none> and password file <none>
I20260809 16:13:45.074918 8742 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:45.074959 8742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:45.076164 8742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:45.076833 8766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.077013 8742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:45.077103 8742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "d0932f8cdd874ee8a871812c33d11bb8"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "2dcdf0db99281b3ea093b893e17c7868"
server_key_iv: "ae9d9de3581f8143722a0353af0348da"
server_key_version: "encryptionkey@0"
I20260809 16:13:45.077415 8742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:45.089253 8742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:45.091318 8742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:45.091455 8742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:45.095381 8742 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:42681
I20260809 16:13:45.095417 8839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:42681 every 8 connection(s)
I20260809 16:13:45.095780 8742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:13:45.096372 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 8742
I20260809 16:13:45.096648 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:37053
--local_ip_for_outbound_sockets=127.16.224.129
--tserver_master_addrs=127.16.224.190:42681
--webserver_port=34247
--webserver_interface=127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:13:45.097438 8840 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:13:45.098100 8840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Bootstrap starting.
I20260809 16:13:45.106612 8840 log.cc:824] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:45.108758 8840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:13:45.109025 8840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8: Bootstrap complete.
I20260809 16:13:45.111202 8840 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:45.111430 8840 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0932f8cdd874ee8a871812c33d11bb8, State: Initialized, Role: FOLLOWER
I20260809 16:13:45.111589 8840 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:45.111666 8840 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:13:45.111716 8840 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:13:45.111760 8840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:13:45.112721 8840 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:45.112819 8840 leader_election.cc:302] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0932f8cdd874ee8a871812c33d11bb8; no voters:
I20260809 16:13:45.112979 8840 leader_election.cc:288] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:13:45.113026 8845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:13:45.113139 8845 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [term 3 LEADER]: Becoming Leader. State: Replica: d0932f8cdd874ee8a871812c33d11bb8, State: Running, Role: LEADER
I20260809 16:13:45.113251 8840 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:13:45.113238 8845 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } }
I20260809 16:13:45.113607 8847 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0932f8cdd874ee8a871812c33d11bb8. Latest consensus state: current_term: 3 leader_uuid: "d0932f8cdd874ee8a871812c33d11bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } } }
I20260809 16:13:45.113932 8847 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:45.114108 8846 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d0932f8cdd874ee8a871812c33d11bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0932f8cdd874ee8a871812c33d11bb8" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42681 } } }
I20260809 16:13:45.114228 8846 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0932f8cdd874ee8a871812c33d11bb8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:13:45.114153 8860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:13:45.115106 8860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:13:45.115335 8860 catalog_manager.cc:1259] Loaded cluster ID: cd0614eaa1df482b8038f661597a1dd9
I20260809 16:13:45.115466 8860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 16:13:45.115751 8860 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9d285729ad google::LogMessage::Fail() at ??:0
@ 0x7f9d2857588a google::LogMessage::SendToLog() at ??:0
@ 0x7f9d285724de google::LogMessage::Flush() at ??:0
@ 0x7f9d28573a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9d304970f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f9d304a08b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9d2a4400d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9d2a435954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9d2b2996db start_thread at ??:0
@ 0x7f9d2707061f clone at ??:0
W20260809 16:13:45.193197 8843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:45.193387 8843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:45.193408 8843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:45.194571 8843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:45.194610 8843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:45.194635 8843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:13:45.196076 8843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:37053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=34247
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:45.196292 8843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:45.196513 8843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:45.202072 8897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:45.202170 8900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:45.202082 8896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:45.202332 8843 server_base.cc:1034] running on GCE node
I20260809 16:13:45.202556 8843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:45.202783 8843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:45.203938 8843 hybrid_clock.cc:648] HybridClock initialized: now 1786292025203927 us; error 26 us; skew 500 ppm
Aug 09 16:13:45 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292025, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:45.206262 8843 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:13:45.206712 8843 webserver.cc:460] Webserver started at http://127.16.224.129:34247/ using document root <none> and password file <none>
I20260809 16:13:45.206943 8843 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:45.206997 8843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:45.208344 8843 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:45.209048 8910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.209226 8843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:45.209326 8843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "260cc7107a574c508754d2f2949b646e"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "822429b7ca46d0b674faefa16859db17"
server_key_iv: "42c1943492d20f183b75ac0f132fc8de"
server_key_version: "encryptionkey@0"
I20260809 16:13:45.209769 8843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:45.231120 8843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:45.233444 8843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:45.233594 8843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:45.233841 8843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:45.234370 8917 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:13:45.235819 8843 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:13:45.235880 8843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:13:45.235915 8843 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:13:45.236807 8843 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:13:45.236852 8843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:13:45.236909 8917 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Bootstrap starting.
I20260809 16:13:45.244230 8843 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:37053
I20260809 16:13:45.244735 8843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:13:45.246621 9085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:37053 every 8 connection(s)
I20260809 16:13:45.247008 8917 log.cc:824] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:13:45.249025 8917 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:13:45.249266 8917 tablet_bootstrap.cc:492] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Bootstrap complete.
I20260809 16:13:45.250066 8917 ts_tablet_manager.cc:1397] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260809 16:13:45.251084 8917 raft_consensus.cc:348] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:45.251308 8917 raft_consensus.cc:729] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260cc7107a574c508754d2f2949b646e, State: Initialized, Role: FOLLOWER
I20260809 16:13:45.251436 8917 consensus_queue.cc:260] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:45.251701 8917 ts_tablet_manager.cc:1428] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260809 16:13:45.253015 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 8843
I20260809 16:13:45.253268 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:35065
--local_ip_for_outbound_sockets=127.16.224.130
--tserver_master_addrs=127.16.224.190:42681
--webserver_port=34841
--webserver_interface=127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:45.302181 9088 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:13:45.245502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:37575)
0809 16:13:45.245725 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.245778 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:45.246510 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:45.302006 (+ 55496us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":116,"threads_started":1}
W20260809 16:13:45.302477 9086 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:42681 (0 consecutive failures): Network error: Failed to ping master at 127.16.224.190:42681: Client connection negotiation failed: client connection to 127.16.224.190:42681: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:13:45.302812 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.302707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:53909)
0809 16:13:45.302753 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.302791 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:13:45.303172 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.303092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:46025)
0809 16:13:45.303133 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.303156 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:13:45.303269 9086 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:13:45.342384 9091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:45.342584 9091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:45.342605 9091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:45.343678 9091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:45.343711 9091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:45.343735 9091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:13:45.345168 9091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:35065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=34841
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:45.345347 9091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:45.345537 9091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:45.350358 9106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:45.350368 9104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:45.350369 9112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:45.350535 9091 server_base.cc:1034] running on GCE node
I20260809 16:13:45.350714 9091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:45.350934 9091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:45.352080 9091 hybrid_clock.cc:648] HybridClock initialized: now 1786292025352063 us; error 28 us; skew 500 ppm
Aug 09 16:13:45 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292025, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:45.354534 9091 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:13:45.354952 9091 webserver.cc:460] Webserver started at http://127.16.224.130:34841/ using document root <none> and password file <none>
I20260809 16:13:45.355130 9091 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:45.355170 9091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:45.356387 9091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:13:45.356911 9122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.357064 9091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.357151 9091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "dfe2f784120c4b2d968e05cefacb5759"
format_stamp: "Formatted at 2026-08-09 16:13:43 on dist-test-slave-fgb3"
server_key: "da50283ed1d6ba8d570a415287a1aa42"
server_key_iv: "4029ed74f659df347a2116c5c7c297bb"
server_key_version: "encryptionkey@0"
I20260809 16:13:45.357553 9091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:45.366987 9091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:45.368589 9091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:45.368708 9091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:45.368920 9091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:45.369196 9091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:45.369228 9091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.369251 9091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:45.369273 9091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.374740 9091 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:35065
I20260809 16:13:45.374814 9293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:35065 every 8 connection(s)
I20260809 16:13:45.375093 9091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:13:45.376055 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.375684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:39767)
0809 16:13:45.375851 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.375923 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
W20260809 16:13:45.376435 9294 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:42681 (0 consecutive failures): Network error: Failed to ping master at 127.16.224.190:42681: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
I20260809 16:13:45.376670 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.376592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:35497)
0809 16:13:45.376630 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.376653 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:13:45.376874 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.376817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:40631)
0809 16:13:45.376846 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.376863 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 16:13:45.376965 9294 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:13:45.378863 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 9091
I20260809 16:13:45.379103 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:34041
--local_ip_for_outbound_sockets=127.16.224.131
--tserver_master_addrs=127.16.224.190:42681
--webserver_port=44355
--webserver_interface=127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:13:45.461194 9298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:13:45.461432 9298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:13:45.461457 9298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:13:45.462657 9298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:13:45.462702 9298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:13:45.462728 9298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:13:45.464221 9298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:34041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=44355
--tserver_master_addrs=127.16.224.190:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:13:45.464455 9298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:13:45.464671 9298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:13:45.469547 9306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:13:45.469532 9312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:45.469636 9298 server_base.cc:1034] running on GCE node
W20260809 16:13:45.469532 9308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:13:45.469895 9298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:13:45.470114 9298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:13:45.471244 9298 hybrid_clock.cc:648] HybridClock initialized: now 1786292025471224 us; error 43 us; skew 500 ppm
Aug 09 16:13:45 dist-test-slave-fgb3 krb5kdc[7065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292025, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:13:45.473587 9298 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:13:45.473989 9298 webserver.cc:460] Webserver started at http://127.16.224.131:44355/ using document root <none> and password file <none>
I20260809 16:13:45.474210 9298 fs_manager.cc:359] Metadata directory not provided
I20260809 16:13:45.474260 9298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:13:45.475538 9298 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:13:45.476060 9321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.476209 9298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:13:45.476293 9298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "49a445597382481796edf17cd8e83f51"
format_stamp: "Formatted at 2026-08-09 16:13:44 on dist-test-slave-fgb3"
server_key: "6a4bad2e5845bdd1f0ed8f4185c61cf5"
server_key_iv: "e517eb1c593059d96ece6cd65486a201"
server_key_version: "encryptionkey@0"
I20260809 16:13:45.476640 9298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:13:45.509418 9298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:13:45.511224 9298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:13:45.511361 9298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:13:45.511595 9298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:13:45.511890 9298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:13:45.511924 9298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.511953 9298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:13:45.511976 9298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:13:45.517666 9298 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:34041
I20260809 16:13:45.517733 9485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:34041 every 8 connection(s)
I20260809 16:13:45.518064 9298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:13:45.519078 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.518711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:36937)
0809 16:13:45.518858 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.518937 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
W20260809 16:13:45.519447 9487 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:42681 (0 consecutive failures): Network error: Failed to ping master at 127.16.224.190:42681: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
I20260809 16:13:45.519670 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.519593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:59479)
0809 16:13:45.519630 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.519654 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 16:13:45.519868 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:45.519810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:48279)
0809 16:13:45.519840 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:45.519857 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 16:13:45.519948 9487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:13:45.524586 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 9298
I20260809 16:13:46.303787 9492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:46.303562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:44145)
0809 16:13:46.303703 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:46.303735 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 16:13:46.377562 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:46.377282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:59617)
0809 16:13:46.377441 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:46.377492 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":58,"threads_started":1}
I20260809 16:13:46.520511 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:46.520269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:36447)
0809 16:13:46.520417 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:46.520456 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260809 16:13:46.937469 9501 raft_consensus.cc:482] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:46.937594 9501 raft_consensus.cc:504] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:46.937937 9501 leader_election.cc:288] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041), dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065)
Aug 09 16:13:46 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292025, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.131@KRBTEST.COM
Aug 09 16:13:46 dist-test-slave-fgb3 krb5kdc[7065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292025, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.130@KRBTEST.COM
I20260809 16:13:46.947795 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:46.938125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:35065 (local address 127.16.224.129:52397)
0809 16:13:46.938242 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:46.938252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:46.938308 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:46.939093 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:46.939096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:46.939445 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:46.939630 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:46.939636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.940694 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.940695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:46.941528 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:46.941531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.942098 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.942099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:46.942196 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:46.942725 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:46.942733 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:46.944605 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:46.946612 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:46.946613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:46.946614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:46.946790 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:46.947006 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:46.947007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:46.947007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:46.947068 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:46.947459 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:46.947461 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:46.947506 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:13:46.947570 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:13:46.947699 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":76,"thread_start_us":53,"threads_started":1}
I20260809 16:13:46.947837 9505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:46.938206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:40309 (local address 127.16.224.131:34041)
0809 16:13:46.938377 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:46.938390 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:46.938424 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:13:46.938871 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:46.938874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:46.938923 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:46.939044 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:46.939753 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.940648 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.941613 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.942004 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.942101 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:46.944322 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:46.944326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:46.944326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:46.944336 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:46.946010 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:46.946492 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:46.946495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:46.946495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:46.946580 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:46.946807 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:46.946808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:46.946808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:46.947090 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:46.947163 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:46.947589 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 16:13:46.947727 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":57,"threads_started":1}
I20260809 16:13:46.947909 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:46.938232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:52397 (local address 127.16.224.130:35065)
0809 16:13:46.938419 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 16:13:46.938429 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:13:46.938446 (+ 17us) server_negotiation.cc:407] Connection header received
0809 16:13:46.938863 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:13:46.938866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:13:46.938905 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:13:46.939043 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:13:46.939749 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.940601 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.941608 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.942023 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.942116 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:46.944713 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:13:46.944716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:13:46.944717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:13:46.944726 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:13:46.946545 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:46.946863 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:46.946865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:46.946865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:46.946949 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:13:46.947132 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:13:46.947133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:13:46.947133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:13:46.947396 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:13:46.947473 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:13:46.947695 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 16:13:46.947816 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":58,"threads_started":1}
I20260809 16:13:46.947854 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:46.938079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:34041 (local address 127.16.224.129:40309)
0809 16:13:46.938228 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:46.938239 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:13:46.938300 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:13:46.939094 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:13:46.939097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:13:46.939445 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:13:46.939630 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:46.939636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.940733 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.940734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:46.941528 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:13:46.941530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:13:46.942087 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:13:46.942088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:13:46.942195 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:13:46.942771 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:13:46.942775 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:13:46.944219 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:13:46.946093 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:46.946100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:46.946102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:46.946416 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:46.946644 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:13:46.946645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:13:46.946645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:13:46.946742 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:13:46.947155 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:13:46.947162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:13:46.947315 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 16:13:46.947450 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 16:13:46.947628 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":49,"thread_start_us":71,"threads_started":1}
W20260809 16:13:46.948915 8911 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
W20260809 16:13:46.948989 8914 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:46.949059 8914 leader_election.cc:302] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 260cc7107a574c508754d2f2949b646e; no voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759
I20260809 16:13:46.949174 9501 raft_consensus.cc:2726] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:13:47.304373 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:47.304241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:40055)
0809 16:13:47.304293 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:47.304330 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:13:47.378278 9507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:47.378029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:59677)
0809 16:13:47.378161 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:47.378233 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260809 16:13:47.521201 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:47.520986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:47199)
0809 16:13:47.521118 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:47.521156 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260809 16:13:48.304997 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:48.304797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:39647)
0809 16:13:48.304915 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:48.304948 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260809 16:13:48.378906 9511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:48.378703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:56785)
0809 16:13:48.378820 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:48.378862 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260809 16:13:48.521833 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:48.521632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:37917)
0809 16:13:48.521750 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:48.521789 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260809 16:13:48.705796 9516 raft_consensus.cc:482] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:48.705907 9516 raft_consensus.cc:504] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:48.706058 9516 leader_election.cc:288] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041), dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065)
W20260809 16:13:48.706512 8914 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
W20260809 16:13:48.706610 8911 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:48.706712 8911 leader_election.cc:302] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 260cc7107a574c508754d2f2949b646e; no voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759
I20260809 16:13:48.706801 9516 raft_consensus.cc:2726] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:13:49.305653 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:49.305421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:33271)
0809 16:13:49.305568 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:49.305604 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260809 16:13:49.379480 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:49.379237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:56827)
0809 16:13:49.379386 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:49.379422 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260809 16:13:49.522557 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:49.522279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:43179)
0809 16:13:49.522481 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:49.522518 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":123,"threads_started":1}
I20260809 16:13:50.306457 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:50.306174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:53467)
0809 16:13:50.306350 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:50.306386 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260809 16:13:50.380101 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:50.379861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:58631)
0809 16:13:50.380014 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:50.380061 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260809 16:13:50.523284 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:50.523046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:46539)
0809 16:13:50.523195 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:50.523236 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
I20260809 16:13:50.876521 9527 raft_consensus.cc:482] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:50.876616 9527 raft_consensus.cc:504] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:50.876812 9527 leader_election.cc:288] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041), dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065)
W20260809 16:13:50.877162 8911 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
W20260809 16:13:50.877254 8914 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:50.877300 8914 leader_election.cc:302] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 260cc7107a574c508754d2f2949b646e; no voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759
I20260809 16:13:50.877383 9527 raft_consensus.cc:2726] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:13:51.307188 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:51.306969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:60995)
0809 16:13:51.307097 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:51.307136 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260809 16:13:51.380685 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:51.380497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:51203)
0809 16:13:51.380607 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:51.380643 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260809 16:13:51.523904 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:51.523692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:46481)
0809 16:13:51.523823 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:51.523860 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260809 16:13:52.307782 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:52.307570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:58135)
0809 16:13:52.307694 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:52.307733 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260809 16:13:52.381349 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:52.381099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:44401)
0809 16:13:52.381263 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:52.381305 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
I20260809 16:13:52.524505 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:52.524280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:57663)
0809 16:13:52.524416 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:52.524457 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260809 16:13:53.308413 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:53.308173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:40831)
0809 16:13:53.308317 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:53.308365 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 16:13:53.382017 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:53.381779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:34077)
0809 16:13:53.381931 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:53.381972 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260809 16:13:53.525130 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:53.524906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:60121)
0809 16:13:53.525051 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:53.525082 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260809 16:13:54.309096 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:54.308861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:46837)
0809 16:13:54.309004 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:54.309046 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260809 16:13:54.382738 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:54.382530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:44979)
0809 16:13:54.382649 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:54.382690 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260809 16:13:54.525853 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:54.525607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:43581)
0809 16:13:54.525752 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:54.525790 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260809 16:13:54.913031 9548 raft_consensus.cc:482] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:54.913144 9548 raft_consensus.cc:504] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:54.913331 9548 leader_election.cc:288] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041), dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065)
W20260809 16:13:54.913712 8914 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
W20260809 16:13:54.913808 8911 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:54.913853 8911 leader_election.cc:302] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 260cc7107a574c508754d2f2949b646e; no voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759
I20260809 16:13:54.913918 9548 raft_consensus.cc:2726] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:13:55.309795 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:55.309590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:56449)
0809 16:13:55.309707 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:55.309745 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260809 16:13:55.383410 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:55.383202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:46311)
0809 16:13:55.383330 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:55.383367 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260809 16:13:55.526537 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:55.526329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:47841)
0809 16:13:55.526454 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:55.526492 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260809 16:13:56.310451 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:56.310215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:60347)
0809 16:13:56.310362 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:56.310401 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260809 16:13:56.384011 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:56.383812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:35757)
0809 16:13:56.383929 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:56.383966 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260809 16:13:56.527168 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:56.526965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:59729)
0809 16:13:56.527085 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:56.527123 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260809 16:13:57.025802 9560 raft_consensus.cc:482] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:13:57.025911 9560 raft_consensus.cc:504] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a445597382481796edf17cd8e83f51" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34041 } } peers { permanent_uuid: "dfe2f784120c4b2d968e05cefacb5759" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35065 } } peers { permanent_uuid: "260cc7107a574c508754d2f2949b646e" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 37053 } }
I20260809 16:13:57.026080 9560 leader_election.cc:288] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041), dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065)
W20260809 16:13:57.026486 8911 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a445597382481796edf17cd8e83f51 (127.16.224.131:34041): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
W20260809 16:13:57.026576 8914 leader_election.cc:341] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfe2f784120c4b2d968e05cefacb5759 (127.16.224.130:35065): Not found: Tablet not found: 7a3003e679da40e2a0a68359fbe926a4
I20260809 16:13:57.026618 8914 leader_election.cc:302] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 260cc7107a574c508754d2f2949b646e; no voters: 49a445597382481796edf17cd8e83f51, dfe2f784120c4b2d968e05cefacb5759
I20260809 16:13:57.026678 9560 raft_consensus.cc:2726] T 7a3003e679da40e2a0a68359fbe926a4 P 260cc7107a574c508754d2f2949b646e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:13:57.311115 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:57.310913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:40311)
0809 16:13:57.311032 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:57.311063 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260809 16:13:57.384605 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:57.384406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:51659)
0809 16:13:57.384521 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:57.384559 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260809 16:13:57.527774 9566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:57.527593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:60323)
0809 16:13:57.527699 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:57.527734 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260809 16:13:58.311754 9567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:58.311546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:43699)
0809 16:13:58.311667 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:58.311704 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260809 16:13:58.385273 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:58.385051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:34639)
0809 16:13:58.385191 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:58.385227 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260809 16:13:58.528453 9569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:58.528202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:37185)
0809 16:13:58.528368 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:58.528407 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260809 16:13:59.312456 9570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:59.312212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:56049)
0809 16:13:59.312352 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:59.312397 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260809 16:13:59.385946 9577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:59.385684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:55449)
0809 16:13:59.385857 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:59.385896 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260809 16:13:59.529129 9578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:13:59.528861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:37095)
0809 16:13:59.529038 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:13:59.529073 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":89,"threads_started":1}
I20260809 16:14:00.313175 9582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:00.312936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.129:36995)
0809 16:14:00.313084 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:00.313123 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260809 16:14:00.386678 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:00.386485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.130:44411)
0809 16:14:00.386598 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:00.386635 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260809 16:14:00.529899 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:00.529657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:42681 (local address 127.16.224.131:33621)
0809 16:14:00.529803 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:00.529847 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260809 16:14:00.530719 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 8843
I20260809 16:14:00.544523 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 9091
I20260809 16:14:00.555480 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 9298
2026-08-09T16:14:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18363 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:14:00 dist-test-slave-fgb3 krb5kdc[9589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:14:00 dist-test-slave-fgb3 krb5kdc[9589](info): set up 2 sockets
Aug 09 16:14:00 dist-test-slave-fgb3 krb5kdc[9589](info): commencing operation
krb5kdc: starting...
W20260809 16:14:01.706913 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:14:01 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:14:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:14:01.823170 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:35019
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:35019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:01.902971 9608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:01.903175 9608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:01.903196 9608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:01.904337 9608 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:14:01.904378 9608 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:14:01.904390 9608 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:14:01.904400 9608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:14:01.904412 9608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:01.904422 9608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:01.904433 9608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:14:01.904444 9608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:14:01.905910 9608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:35019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:35019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:01.906157 9608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:01.906399 9608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:01.911191 9621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:01.911196 9617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:01.911383 9608 server_base.cc:1034] running on GCE node
W20260809 16:14:01.911255 9619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:01.911651 9608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:01.911929 9608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:01.913075 9608 hybrid_clock.cc:648] HybridClock initialized: now 1786292041913063 us; error 29 us; skew 500 ppm
Aug 09 16:14:01 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292041, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:01.915500 9608 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:14:01.915926 9608 webserver.cc:460] Webserver started at http://127.16.224.190:43113/ using document root <none> and password file <none>
I20260809 16:14:01.916160 9608 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:01.916208 9608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:01.916324 9608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:14:01.917438 9608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "907c31399a4b45afb99db75efe84bd0d"
format_stamp: "Formatted at 2026-08-09 16:14:01 on dist-test-slave-fgb3"
server_key: "03e78414f57dfc642738a863bf6a10e3"
server_key_iv: "285f9a62aac474bba83ca7fcfb4ef558"
server_key_version: "encryptionkey@0"
I20260809 16:14:01.917783 9608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "907c31399a4b45afb99db75efe84bd0d"
format_stamp: "Formatted at 2026-08-09 16:14:01 on dist-test-slave-fgb3"
server_key: "03e78414f57dfc642738a863bf6a10e3"
server_key_iv: "285f9a62aac474bba83ca7fcfb4ef558"
server_key_version: "encryptionkey@0"
I20260809 16:14:01.919090 9608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:14:01.919739 9628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:01.919891 9608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:01.919983 9608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "907c31399a4b45afb99db75efe84bd0d"
format_stamp: "Formatted at 2026-08-09 16:14:01 on dist-test-slave-fgb3"
server_key: "03e78414f57dfc642738a863bf6a10e3"
server_key_iv: "285f9a62aac474bba83ca7fcfb4ef558"
server_key_version: "encryptionkey@0"
I20260809 16:14:01.920050 9608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:01.933903 9608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:01.935760 9608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:01.935874 9608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:01.939759 9608 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:35019
I20260809 16:14:01.939802 9709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:35019 every 8 connection(s)
I20260809 16:14:01.940119 9608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:14:01.940968 9710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:14:01.943327 9710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Bootstrap starting.
I20260809 16:14:01.943888 9710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Neither blocks nor log segments found. Creating new log.
I20260809 16:14:01.944128 9710 log.cc:824] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:01.944725 9710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: No bootstrap required, opened a new log
I20260809 16:14:01.946259 9710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:01.946455 9710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:14:01.946488 9710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907c31399a4b45afb99db75efe84bd0d, State: Initialized, Role: FOLLOWER
I20260809 16:14:01.946595 9710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:01.946645 9710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:14:01.946669 9710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:14:01.946700 9710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:14:01.947206 9710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:01.947295 9710 leader_election.cc:302] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907c31399a4b45afb99db75efe84bd0d; no voters:
I20260809 16:14:01.947423 9710 leader_election.cc:288] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:14:01.947482 9715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:14:01.947629 9715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 LEADER]: Becoming Leader. State: Replica: 907c31399a4b45afb99db75efe84bd0d, State: Running, Role: LEADER
I20260809 16:14:01.947682 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 9608
I20260809 16:14:01.947687 9710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:14:01.947790 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:14:01.947790 9715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:01.948009 17282 external_mini_cluster.cc:1428] Setting key 29cdae3edf57d64e0d12824995403ac9
I20260809 16:14:01.948232 9717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907c31399a4b45afb99db75efe84bd0d. Latest consensus state: current_term: 1 leader_uuid: "907c31399a4b45afb99db75efe84bd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } } }
I20260809 16:14:01.948318 9717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: This master's current role is: LEADER
I20260809 16:14:01.948243 9716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "907c31399a4b45afb99db75efe84bd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } } }
I20260809 16:14:01.948630 9716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: This master's current role is: LEADER
I20260809 16:14:01.949029 9725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:14:01.949612 9725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:14:01.951275 9725 catalog_manager.cc:1347] Generated new cluster ID: 7f0a02396b344c4bba3f95a8126821af
I20260809 16:14:01.951330 9725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:14:01 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:14:01.959388 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:01.948682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57802 (local address 127.16.224.190:35019)
0809 16:14:01.948871 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:01.948881 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:01.948987 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:14:01.949210 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:01.949219 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:01.949283 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:01.950065 (+ 782us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:01.950144 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:01.951315 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:01.952166 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:01.952583 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:01.952679 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:01.954924 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:01.954929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:01.954929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:01.954941 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:01.957620 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:01.958064 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:01.958068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:01.958068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:01.958197 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:01.958394 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:01.958394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:01.958395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:01.958728 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:01.958883 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:01.958974 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:14:01.959127 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260809 16:14:01.968591 9725 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:14:01.969370 9725 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:14:01.974937 9725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Generated new TSK 0
I20260809 16:14:01.975216 9725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:14:02.001452 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:35019
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:02.084913 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:02.085129 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:02.085150 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:02.086344 9748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:02.086381 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:02.086410 9748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:14:02.087889 9748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:02.088129 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:02.088347 9748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:02.093488 9757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:02.093497 9756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:02.093497 9759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:02.093737 9748 server_base.cc:1034] running on GCE node
I20260809 16:14:02.093932 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:02.094177 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:02.095319 9748 hybrid_clock.cc:648] HybridClock initialized: now 1786292042095298 us; error 37 us; skew 500 ppm
Aug 09 16:14:02 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:02.097834 9748 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:14:02.098295 9748 webserver.cc:460] Webserver started at http://127.16.224.129:40301/ using document root <none> and password file <none>
I20260809 16:14:02.098552 9748 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:02.098600 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:02.098731 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:14:02.099747 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "545ebbee9c8a4a7a8e7b39352dce92b9"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "4cbde485e3f470198e11b1cf2aeb0623"
server_key_iv: "1c748487f8cdbe7bda2abc2582dce630"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.100099 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "545ebbee9c8a4a7a8e7b39352dce92b9"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "4cbde485e3f470198e11b1cf2aeb0623"
server_key_iv: "1c748487f8cdbe7bda2abc2582dce630"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.101318 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:14:02.101945 9769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.102114 9748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:14:02.102187 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "545ebbee9c8a4a7a8e7b39352dce92b9"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "4cbde485e3f470198e11b1cf2aeb0623"
server_key_iv: "1c748487f8cdbe7bda2abc2582dce630"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.102258 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:02.110714 9748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:02.112709 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:02.112843 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:02.113052 9748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:02.113338 9748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:14:02.113371 9748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.113397 9748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:14:02.113413 9748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.118964 9748 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:36107
I20260809 16:14:02.119025 9948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:36107 every 8 connection(s)
I20260809 16:14:02.119297 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:14:02 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:14:02.126899 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 9748
I20260809 16:14:02.127005 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:14:02.127207 17282 external_mini_cluster.cc:1428] Setting key 6697ceafc9de5a33a43b9be500c12c09
I20260809 16:14:02.127930 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.119908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:43765 (local address 127.16.224.190:35019)
0809 16:14:02.119958 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.119960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.120147 (+ 187us) server_negotiation.cc:407] Connection header received
0809 16:14:02.120651 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.120653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.120670 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.120722 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:02.121634 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.122167 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.123209 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.123424 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.123489 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.125412 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:02.125414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:02.125414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:02.125423 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:02.126564 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.126993 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.126994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.126994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.127053 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.127298 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.127299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.127300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.127428 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:02.127502 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.127769 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.127862 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:14:02.128144 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.119857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:43765)
0809 16:14:02.120032 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.120065 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.120540 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.120787 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.120790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.121349 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:02.121535 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.121540 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.122254 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.122255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.123131 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.123134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.123497 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.123498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.123611 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.124040 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:02.124048 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:02.125329 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:02.126637 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:02.126643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:02.126645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:02.126912 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:02.127120 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:02.127121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:02.127122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:02.127224 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:02.127493 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:02.127499 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:02.127643 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.127748 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.127909 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260809 16:14:02.129866 9951 heartbeater.cc:347] Connected to a master server at 127.16.224.190:35019
I20260809 16:14:02.129966 9951 heartbeater.cc:464] Registering TS with master...
I20260809 16:14:02.130134 9951 heartbeater.cc:511] Master 127.16.224.190:35019 requested a full tablet report, sending...
I20260809 16:14:02.130700 9655 ts_manager.cc:194] Registered new tserver with Master: 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129:36107)
I20260809 16:14:02.131496 9655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:43765
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:14:02.150981 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:35019
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:02.233364 9957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:02.233560 9957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:02.233592 9957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:02.234687 9957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:02.234724 9957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:02.234758 9957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:14:02.236128 9957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:02.236342 9957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:02.236550 9957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:02.241153 9971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:02.241261 9968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:02.241153 9967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:02.241529 9957 server_base.cc:1034] running on GCE node
I20260809 16:14:02.241714 9957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:02.241916 9957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:02.243046 9957 hybrid_clock.cc:648] HybridClock initialized: now 1786292042243026 us; error 30 us; skew 500 ppm
Aug 09 16:14:02 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:02.245327 9957 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:14:02.245724 9957 webserver.cc:460] Webserver started at http://127.16.224.130:46335/ using document root <none> and password file <none>
I20260809 16:14:02.245889 9957 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:02.245924 9957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:02.246013 9957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:14:02.246955 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "462e502be3d845da872817ceb9f54ef6"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "1720b409fe437776b76cb7f5a1c62bc3"
server_key_iv: "039af7ac391d1c5ee5fdd2a6cb51b175"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.247237 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "462e502be3d845da872817ceb9f54ef6"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "1720b409fe437776b76cb7f5a1c62bc3"
server_key_iv: "039af7ac391d1c5ee5fdd2a6cb51b175"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.248432 9957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:14:02.249126 9980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.249284 9957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:14:02.249364 9957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "462e502be3d845da872817ceb9f54ef6"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "1720b409fe437776b76cb7f5a1c62bc3"
server_key_iv: "039af7ac391d1c5ee5fdd2a6cb51b175"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.249434 9957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:02.265204 9957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:02.267145 9957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:02.267274 9957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:02.267535 9957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:02.267868 9957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:14:02.267904 9957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.267935 9957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:14:02.267952 9957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.273501 9957 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:46187
I20260809 16:14:02.273587 10150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:46187 every 8 connection(s)
I20260809 16:14:02.273896 9957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:14:02.279038 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 9957
I20260809 16:14:02.279124 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:14:02.279309 17282 external_mini_cluster.cc:1428] Setting key 3d0a9e23d4695d5c9d469ddf8bec01e9
Aug 09 16:14:02 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:14:02.282909 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.274655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:50963 (local address 127.16.224.190:35019)
0809 16:14:02.274714 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.274716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.274911 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:14:02.275389 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.275390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.275400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.275466 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:02.276355 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.276863 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.277860 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.278085 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.278138 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.280268 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:02.280270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:02.280270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:02.280278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:02.281556 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.281958 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.281960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.281960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.282020 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.282240 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.282241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.282241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.282402 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:02.282461 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.282721 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.282819 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:14:02.283054 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.274622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:50963)
0809 16:14:02.274785 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.274828 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.275311 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.275523 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.275526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.276091 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:02.276286 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.276292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.276932 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.276932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.277760 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.277763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.278151 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.278152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.278252 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.278691 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:02.278699 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:02.280179 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:02.281648 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:02.281656 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:02.281658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:02.281876 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:02.282079 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:02.282080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:02.282080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:02.282179 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:02.282466 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:02.282473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:02.282599 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.282690 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.282840 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260809 16:14:02.283496 10151 heartbeater.cc:347] Connected to a master server at 127.16.224.190:35019
I20260809 16:14:02.283600 10151 heartbeater.cc:464] Registering TS with master...
I20260809 16:14:02.283779 10151 heartbeater.cc:511] Master 127.16.224.190:35019 requested a full tablet report, sending...
I20260809 16:14:02.284171 9655 ts_manager.cc:194] Registered new tserver with Master: 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
I20260809 16:14:02.284682 9655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:50963
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:14:02.304342 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:35019
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:02.385509 10157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:02.385704 10157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:02.385725 10157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:02.386861 10157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:02.386904 10157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:02.386929 10157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:14:02.388398 10157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:02.388629 10157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:02.388857 10157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:02.393622 10166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:02.393751 10157 server_base.cc:1034] running on GCE node
W20260809 16:14:02.393604 10168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:02.393659 10165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:02.394007 10157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:02.394258 10157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:02.395393 10157 hybrid_clock.cc:648] HybridClock initialized: now 1786292042395396 us; error 43 us; skew 500 ppm
Aug 09 16:14:02 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:02.397756 10157 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:14:02.398212 10157 webserver.cc:460] Webserver started at http://127.16.224.131:33747/ using document root <none> and password file <none>
I20260809 16:14:02.398453 10157 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:02.398504 10157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:02.398633 10157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:14:02.399668 10157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "50966eca2d4f4bdd8f34a32812e61849"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "919d59b4ef88f3942a2d580f2c51e1e6"
server_key_iv: "9383995e212ae6230413f2cec90648d2"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.400022 10157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "50966eca2d4f4bdd8f34a32812e61849"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "919d59b4ef88f3942a2d580f2c51e1e6"
server_key_iv: "9383995e212ae6230413f2cec90648d2"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.401247 10157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:14:02.401880 10178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.402062 10157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:14:02.402150 10157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "50966eca2d4f4bdd8f34a32812e61849"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "919d59b4ef88f3942a2d580f2c51e1e6"
server_key_iv: "9383995e212ae6230413f2cec90648d2"
server_key_version: "encryptionkey@0"
I20260809 16:14:02.402210 10157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:02.413898 10157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:02.416137 10157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:02.416286 10157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:02.416518 10157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:02.416846 10157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:14:02.416890 10157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.416922 10157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:14:02.416939 10157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:02.422650 10157 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:44465
I20260809 16:14:02.422703 10354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:44465 every 8 connection(s)
I20260809 16:14:02.423017 10157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:14:02 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:14:02.429395 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 10157
I20260809 16:14:02.429500 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:14:02.429716 17282 external_mini_cluster.cc:1428] Setting key bbb7739ec5a2d9be00077225067bcbcc
I20260809 16:14:02.431842 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.423734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:47613 (local address 127.16.224.190:35019)
0809 16:14:02.423798 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.423801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.423994 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:14:02.424469 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.424470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.424480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.424525 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:02.425403 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.425897 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.426871 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.427047 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.427092 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.429069 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:02.429071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:02.429071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:02.429078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:02.430599 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.430952 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.430953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.430954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.431015 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.431233 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.431234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.431235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.431355 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:02.431411 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.431671 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.431753 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:14:02.432017 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.423716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:47613)
0809 16:14:02.423871 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.423909 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.424383 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.424591 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.424594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.425140 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:02.425334 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.425340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.425963 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.425964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.426808 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.426811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.427104 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.427105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.427205 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.427667 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:02.427675 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:02.428985 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:02.430658 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:02.430666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:02.430668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:02.430879 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:02.431072 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:02.431072 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:02.431073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:02.431172 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:02.431411 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:02.431418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:02.431548 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.431636 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.431787 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260809 16:14:02.432451 10357 heartbeater.cc:347] Connected to a master server at 127.16.224.190:35019
I20260809 16:14:02.432539 10357 heartbeater.cc:464] Registering TS with master...
I20260809 16:14:02.432708 10357 heartbeater.cc:511] Master 127.16.224.190:35019 requested a full tablet report, sending...
I20260809 16:14:02.433070 9655 ts_manager.cc:194] Registered new tserver with Master: 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465)
I20260809 16:14:02.433425 9655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:47613
I20260809 16:14:02.441470 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:14:02.448429 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.442825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.16.224.190:35019)
0809 16:14:02.442875 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.442877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.443007 (+ 130us) server_negotiation.cc:407] Connection header received
0809 16:14:02.443068 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.443069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.443080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.443124 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:02.443674 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.444152 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.444918 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.445106 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.445159 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.446224 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:02.446225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:02.446226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:02.446233 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:02.447271 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.447618 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.447619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.447620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.447674 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:02.447837 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:02.447838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:02.447839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:02.448039 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:02.448132 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.448259 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.448338 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:14:02.449887 9655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:14:02.451493 9655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:14:02.457835 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.453547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44465 (local address 127.0.0.1:57008)
0809 16:14:02.453978 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.454022 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.454102 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.454307 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.454341 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.454343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:02.454588 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.454591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.455927 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.455929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.456895 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.456897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.457536 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.457537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.457617 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.457628 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.457688 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.457749 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":28,"thread_start_us":66,"threads_started":1}
I20260809 16:14:02.457839 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.453804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57008 (local address 127.16.224.131:44465)
0809 16:14:02.453962 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.453966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.454104 (+ 138us) server_negotiation.cc:407] Connection header received
0809 16:14:02.454189 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.454191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.454205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.454282 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:02.454671 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.455805 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.456970 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.457449 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.457514 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.457570 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.457713 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.457757 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260809 16:14:02.457880 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.453734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.16.224.129:36107)
0809 16:14:02.453940 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.453945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.454244 (+ 299us) server_negotiation.cc:407] Connection header received
0809 16:14:02.454298 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.454300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.454332 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.454406 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:02.454573 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.455314 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.456648 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.457379 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.457445 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.457502 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.457742 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.457791 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":96,"threads_started":1}
I20260809 16:14:02.458032 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.453551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46187 (local address 127.0.0.1:35132)
0809 16:14:02.454265 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.454272 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.454409 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.454686 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.454687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.454689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:02.454722 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.454723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.455936 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.455937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.456913 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.456916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.457798 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.457798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.457854 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.457858 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.457902 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.457939 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":28,"thread_start_us":53,"threads_started":1}
I20260809 16:14:02.458045 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.453547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:36107 (local address 127.0.0.1:49620)
0809 16:14:02.454183 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.454190 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.454223 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.454408 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.454409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.454411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:02.454490 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.454492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.455411 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.455412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.456581 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.456584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.457538 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.457539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.457632 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.457637 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.457691 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.457964 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":230,"spinlock_wait_cycles":896,"thread_start_us":64,"threads_started":1}
I20260809 16:14:02.458274 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.453794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.16.224.130:46187)
0809 16:14:02.454052 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.454055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.454386 (+ 331us) server_negotiation.cc:407] Connection header received
0809 16:14:02.454518 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.454521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.454537 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.454607 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:02.454791 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.455836 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.456999 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.457666 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.457720 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.457767 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.458179 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.458214 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":88,"threads_started":1}
I20260809 16:14:02.458913 9845 tablet_service.cc:1467] Processing CreateTablet for tablet 687f71bca96846b2ae24ca1a81f1d310 (DEFAULT_TABLE table=test-table [id=3ec72bde1d3d4d9d9acaeb5a4b128b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:14:02.458935 10053 tablet_service.cc:1467] Processing CreateTablet for tablet 687f71bca96846b2ae24ca1a81f1d310 (DEFAULT_TABLE table=test-table [id=3ec72bde1d3d4d9d9acaeb5a4b128b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:14:02.459276 9845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687f71bca96846b2ae24ca1a81f1d310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:14:02.459295 10053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687f71bca96846b2ae24ca1a81f1d310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:14:02.461767 10379 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap starting.
I20260809 16:14:02.462031 10380 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Bootstrap starting.
I20260809 16:14:02.462519 10379 tablet_bootstrap.cc:654] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Neither blocks nor log segments found. Creating new log.
I20260809 16:14:02.462473 10254 tablet_service.cc:1467] Processing CreateTablet for tablet 687f71bca96846b2ae24ca1a81f1d310 (DEFAULT_TABLE table=test-table [id=3ec72bde1d3d4d9d9acaeb5a4b128b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:14:02.462628 10380 tablet_bootstrap.cc:654] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Neither blocks nor log segments found. Creating new log.
I20260809 16:14:02.462810 10254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 687f71bca96846b2ae24ca1a81f1d310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:14:02.463100 10380 log.cc:824] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:02.463099 10379 log.cc:824] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:02.463879 10380 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: No bootstrap required, opened a new log
I20260809 16:14:02.463954 10380 ts_tablet_manager.cc:1397] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:14:02.465945 10385 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Bootstrap starting.
I20260809 16:14:02.465858 10380 raft_consensus.cc:348] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.466091 10380 raft_consensus.cc:374] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:14:02.466142 10380 raft_consensus.cc:729] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462e502be3d845da872817ceb9f54ef6, State: Initialized, Role: FOLLOWER
I20260809 16:14:02.466253 10380 consensus_queue.cc:260] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.466539 10380 ts_tablet_manager.cc:1428] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 16:14:02.466622 10385 tablet_bootstrap.cc:654] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Neither blocks nor log segments found. Creating new log.
I20260809 16:14:02.466678 10151 heartbeater.cc:503] Master 127.16.224.190:35019 was elected leader, sending a full tablet report...
I20260809 16:14:02.466969 10379 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: No bootstrap required, opened a new log
I20260809 16:14:02.467039 10379 ts_tablet_manager.cc:1397] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 16:14:02.467167 10385 log.cc:824] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:02.467833 10385 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: No bootstrap required, opened a new log
I20260809 16:14:02.467885 10385 ts_tablet_manager.cc:1397] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:14:02.468619 10379 raft_consensus.cc:348] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.468753 10379 raft_consensus.cc:374] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:14:02.468794 10379 raft_consensus.cc:729] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545ebbee9c8a4a7a8e7b39352dce92b9, State: Initialized, Role: FOLLOWER
I20260809 16:14:02.468906 10379 consensus_queue.cc:260] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.469157 10379 ts_tablet_manager.cc:1428] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:14:02.469244 9951 heartbeater.cc:503] Master 127.16.224.190:35019 was elected leader, sending a full tablet report...
I20260809 16:14:02.469192 10385 raft_consensus.cc:348] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.469347 10385 raft_consensus.cc:374] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:14:02.469381 10385 raft_consensus.cc:729] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50966eca2d4f4bdd8f34a32812e61849, State: Initialized, Role: FOLLOWER
I20260809 16:14:02.469506 10385 consensus_queue.cc:260] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.469775 10385 ts_tablet_manager.cc:1428] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:14:02.469796 10357 heartbeater.cc:503] Master 127.16.224.190:35019 was elected leader, sending a full tablet report...
W20260809 16:14:02.524458 10152 tablet.cc:2369] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:14:02.606282 10390 raft_consensus.cc:482] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:02.606437 10390 raft_consensus.cc:504] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.606832 10390 leader_election.cc:288] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
I20260809 16:14:02.609483 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.607119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:41983 (local address 127.16.224.130:46187)
0809 16:14:02.607173 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.607175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.607183 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:14:02.607210 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.607211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.607218 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.607264 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:02.607410 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.607876 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.608792 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.609167 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.609229 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.609239 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.609384 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.609417 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:14:02.609484 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.607006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46187 (local address 127.16.224.129:41983)
0809 16:14:02.607075 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.607085 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.607125 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.607276 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.607278 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.607279 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:02.607346 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.607348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.607948 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.607950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.608698 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.608700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.609242 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.609243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.609314 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.609318 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.609363 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.609404 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":18}
I20260809 16:14:02.609552 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.607077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:47791 (local address 127.16.224.131:44465)
0809 16:14:02.607134 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.607136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.607270 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:14:02.607325 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.607326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.607335 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.607375 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:02.607531 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.607981 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.608871 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.609230 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.609290 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.609298 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.609447 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.609489 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:14:02.609611 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.607007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44465 (local address 127.16.224.129:47791)
0809 16:14:02.607200 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.607208 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:02.607265 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:02.607386 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:02.607387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:02.607388 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:02.607465 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.607467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.608055 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.608056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.608786 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:02.608787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.609306 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.609307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:02.609378 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.609381 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:14:02.609426 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:14:02.609464 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":44,"thread_start_us":82,"threads_started":1}
I20260809 16:14:02.609812 10086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687f71bca96846b2ae24ca1a81f1d310" candidate_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462e502be3d845da872817ceb9f54ef6" is_pre_election: true
I20260809 16:14:02.609948 10086 raft_consensus.cc:2454] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 545ebbee9c8a4a7a8e7b39352dce92b9 in term 0.
I20260809 16:14:02.610178 9778 leader_election.cc:302] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 462e502be3d845da872817ceb9f54ef6, 545ebbee9c8a4a7a8e7b39352dce92b9; no voters:
I20260809 16:14:02.610245 10281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687f71bca96846b2ae24ca1a81f1d310" candidate_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50966eca2d4f4bdd8f34a32812e61849" is_pre_election: true
I20260809 16:14:02.610351 10390 raft_consensus.cc:2781] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:14:02.610424 10390 raft_consensus.cc:482] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:14:02.610435 10281 raft_consensus.cc:2454] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 545ebbee9c8a4a7a8e7b39352dce92b9 in term 0.
I20260809 16:14:02.610459 10390 raft_consensus.cc:3037] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:14:02.611199 10390 raft_consensus.cc:504] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.611322 10390 leader_election.cc:288] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 1 election: Requested vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
I20260809 16:14:02.611479 10086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687f71bca96846b2ae24ca1a81f1d310" candidate_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462e502be3d845da872817ceb9f54ef6"
I20260809 16:14:02.611558 10086 raft_consensus.cc:3037] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:14:02.611554 10281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "687f71bca96846b2ae24ca1a81f1d310" candidate_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50966eca2d4f4bdd8f34a32812e61849"
I20260809 16:14:02.611630 10281 raft_consensus.cc:3037] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:14:02.612272 10086 raft_consensus.cc:2454] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 545ebbee9c8a4a7a8e7b39352dce92b9 in term 1.
I20260809 16:14:02.612352 10281 raft_consensus.cc:2454] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 545ebbee9c8a4a7a8e7b39352dce92b9 in term 1.
I20260809 16:14:02.612432 9778 leader_election.cc:302] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 462e502be3d845da872817ceb9f54ef6, 545ebbee9c8a4a7a8e7b39352dce92b9; no voters:
I20260809 16:14:02.612531 10390 raft_consensus.cc:2781] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:14:02.612685 10390 raft_consensus.cc:686] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 LEADER]: Becoming Leader. State: Replica: 545ebbee9c8a4a7a8e7b39352dce92b9, State: Running, Role: LEADER
I20260809 16:14:02.612783 10390 consensus_queue.cc:237] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:02.613655 9655 catalog_manager.cc:5495] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129). New cstate: current_term: 1 leader_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } health_report { overall_health: UNKNOWN } } }
W20260809 16:14:02.619982 9952 tablet.cc:2369] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:14:02.637713 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.635379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.16.224.129:36107)
0809 16:14:02.635439 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:02.635441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:02.635449 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:14:02.635526 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:02.635527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:02.635547 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:02.635605 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:14:02.635718 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.636184 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.636981 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:02.637157 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:02.637202 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:02.637416 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:14:02.637522 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:14:02.637577 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:02.637605 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:14:02.637620 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:14:02.639546 10086 raft_consensus.cc:1261] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Refusing update from remote peer 545ebbee9c8a4a7a8e7b39352dce92b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:14:02.639550 10281 raft_consensus.cc:1261] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 1 FOLLOWER]: Refusing update from remote peer 545ebbee9c8a4a7a8e7b39352dce92b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:14:02.639894 10390 consensus_queue.cc:1035] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:14:02.640045 10395 consensus_queue.cc:1035] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:14:02.644708 9653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:14:02.644795 9653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:14:02.645529 9653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Sending DeleteTablet for 3 replicas of tablet 687f71bca96846b2ae24ca1a81f1d310
I20260809 16:14:02.645853 10254 tablet_service.cc:1514] Processing DeleteTablet for tablet 687f71bca96846b2ae24ca1a81f1d310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:14:02 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:57008
I20260809 16:14:02.645895 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 9748
I20260809 16:14:02.645918 10053 tablet_service.cc:1514] Processing DeleteTablet for tablet 687f71bca96846b2ae24ca1a81f1d310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:14:02 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:35132
I20260809 16:14:02.645954 9845 tablet_service.cc:1514] Processing DeleteTablet for tablet 687f71bca96846b2ae24ca1a81f1d310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:14:02 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:49620
I20260809 16:14:02.648628 10414 tablet_replica.cc:330] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: stopping tablet replica
I20260809 16:14:02.648712 10414 raft_consensus.cc:2229] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:14:02.648785 10414 raft_consensus.cc:2258] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:14:02.650974 10414 ts_tablet_manager.cc:1905] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:14:02.652549 10414 ts_tablet_manager.cc:1918] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:14:02.652611 10414 log.cc:1196] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/687f71bca96846b2ae24ca1a81f1d310
I20260809 16:14:02.652850 10414 ts_tablet_manager.cc:1939] T 687f71bca96846b2ae24ca1a81f1d310 P 50966eca2d4f4bdd8f34a32812e61849: Deleting consensus metadata
I20260809 16:14:02.653290 9631 catalog_manager.cc:4841] TS 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): tablet 687f71bca96846b2ae24ca1a81f1d310 (table test-table [id=3ec72bde1d3d4d9d9acaeb5a4b128b7e]) successfully deleted
I20260809 16:14:02.654793 10415 tablet_replica.cc:330] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: stopping tablet replica
I20260809 16:14:02.654909 10415 raft_consensus.cc:2229] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:14:02.655014 10415 raft_consensus.cc:2258] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:14:02.655428 10415 ts_tablet_manager.cc:1905] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:14:02.656209 9630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:36107 (error 108)
I20260809 16:14:02.656489 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 9957
I20260809 16:14:02.661794 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.656572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:36107 (local address 127.0.0.1:49644)
0809 16:14:02.661704 (+ 5132us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.661759 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:36107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5106}
W20260809 16:14:02.661949 9630 catalog_manager.cc:4584] TS 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129:36107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 687f71bca96846b2ae24ca1a81f1d310: Network error: Client connection negotiation failed: client connection to 127.16.224.129:36107: connect: Connection refused (error 111)
I20260809 16:14:02.671772 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:02.671635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46187 (local address 127.0.0.1:35146)
0809 16:14:02.671694 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:02.671733 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.130:46187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:14:02.671748 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 10157
I20260809 16:14:02.687795 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 9608
I20260809 16:14:22.698987 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:35019
--webserver_interface=127.16.224.190
--webserver_port=43113
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:35019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:22.781339 10420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:22.781543 10420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:22.781572 10420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:22.782723 10420 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:14:22.782761 10420 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:14:22.782778 10420 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:14:22.782794 10420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:14:22.782809 10420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:22.782827 10420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:22.782845 10420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:14:22.782868 10420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:14:22.784376 10420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:35019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:35019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:22.784610 10420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:22.784825 10420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:22.793903 10437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:22.793985 10420 server_base.cc:1034] running on GCE node
W20260809 16:14:22.794005 10434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:22.793926 10433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:22.794538 10420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:22.794835 10420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:22.795984 10420 hybrid_clock.cc:648] HybridClock initialized: now 1786292062795975 us; error 36 us; skew 500 ppm
Aug 09 16:14:22 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292062, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:22.808100 10420 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:14:22.808621 10420 webserver.cc:460] Webserver started at http://127.16.224.190:43113/ using document root <none> and password file <none>
I20260809 16:14:22.808866 10420 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:22.808926 10420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:22.810345 10420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:14:22.810983 10446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:22.811146 10420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:14:22.811242 10420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "907c31399a4b45afb99db75efe84bd0d"
format_stamp: "Formatted at 2026-08-09 16:14:01 on dist-test-slave-fgb3"
server_key: "03e78414f57dfc642738a863bf6a10e3"
server_key_iv: "285f9a62aac474bba83ca7fcfb4ef558"
server_key_version: "encryptionkey@0"
I20260809 16:14:22.811671 10420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:22.834040 10420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:22.836237 10420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:22.836356 10420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:22.840389 10420 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:35019
I20260809 16:14:22.840427 10529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:35019 every 8 connection(s)
I20260809 16:14:22.840816 10420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:14:22.842417 10530 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:14:22.843139 10530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Bootstrap starting.
I20260809 16:14:22.844836 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 10420
I20260809 16:14:22.845099 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:36107
--local_ip_for_outbound_sockets=127.16.224.129
--tserver_master_addrs=127.16.224.190:35019
--webserver_port=40301
--webserver_interface=127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:14:22.852317 10530 log.cc:824] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:22.854305 10530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:14:22.854637 10530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Bootstrap complete.
I20260809 16:14:22.856822 10530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:22.857074 10530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907c31399a4b45afb99db75efe84bd0d, State: Initialized, Role: FOLLOWER
I20260809 16:14:22.857240 10530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:22.857321 10530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:14:22.857378 10530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:14:22.857427 10530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:14:22.858500 10530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:22.858604 10530 leader_election.cc:302] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907c31399a4b45afb99db75efe84bd0d; no voters:
I20260809 16:14:22.858763 10530 leader_election.cc:288] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:14:22.858812 10541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:14:22.858979 10541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 LEADER]: Becoming Leader. State: Replica: 907c31399a4b45afb99db75efe84bd0d, State: Running, Role: LEADER
I20260809 16:14:22.859055 10530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:14:22.859129 10541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:22.859457 10542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "907c31399a4b45afb99db75efe84bd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } } }
I20260809 16:14:22.859534 10542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: This master's current role is: LEADER
I20260809 16:14:22.859755 10545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907c31399a4b45afb99db75efe84bd0d. Latest consensus state: current_term: 2 leader_uuid: "907c31399a4b45afb99db75efe84bd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } } }
I20260809 16:14:22.859869 10545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: This master's current role is: LEADER
I20260809 16:14:22.859990 10550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:14:22.860854 10550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:14:22.861022 10550 catalog_manager.cc:1259] Loaded cluster ID: 7f0a02396b344c4bba3f95a8126821af
I20260809 16:14:22.861238 10550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:14:22.862040 10550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:14:22.862221 10550 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Loaded TSK: 0
I20260809 16:14:22.872444 10550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Generated new TSK 1
I20260809 16:14:22.872545 10550 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:14:22.943051 10534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:22.943248 10534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:22.943270 10534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:22.944394 10534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:22.944442 10534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:22.944468 10534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:14:22.945894 10534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:36107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=40301
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:22.946101 10534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:22.946336 10534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:22.952042 10579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:22.952093 10576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:22.952286 10582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:22.952368 10534 server_base.cc:1034] running on GCE node
I20260809 16:14:22.952597 10534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:22.952889 10534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:22.954077 10534 hybrid_clock.cc:648] HybridClock initialized: now 1786292062954057 us; error 33 us; skew 500 ppm
Aug 09 16:14:22 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292062, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:22.957018 10534 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:14:22.957588 10534 webserver.cc:460] Webserver started at http://127.16.224.129:40301/ using document root <none> and password file <none>
I20260809 16:14:22.957829 10534 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:22.957912 10534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:22.959661 10534 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:14:22.960397 10596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:22.960567 10534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:14:22.960655 10534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "545ebbee9c8a4a7a8e7b39352dce92b9"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "4cbde485e3f470198e11b1cf2aeb0623"
server_key_iv: "1c748487f8cdbe7bda2abc2582dce630"
server_key_version: "encryptionkey@0"
I20260809 16:14:22.960983 10534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:22.980149 10534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:22.982358 10534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:22.982489 10534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:22.982729 10534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:22.983186 10605 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:14:22.985232 10534 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:14:22.985275 10534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:14:22.985306 10534 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:14:22.985929 10534 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:14:22.985970 10534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:14:22.986059 10605 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap starting.
I20260809 16:14:22.993964 10534 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:36107
I20260809 16:14:22.994302 10776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:36107 every 8 connection(s)
I20260809 16:14:22.994479 10534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:14:22.997433 10605 log.cc:824] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:23.000269 10605 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:14:23.000697 10605 tablet_bootstrap.cc:492] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap complete.
I20260809 16:14:23.001665 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 10534
I20260809 16:14:23.001907 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:46187
--local_ip_for_outbound_sockets=127.16.224.130
--tserver_master_addrs=127.16.224.190:35019
--webserver_port=46335
--webserver_interface=127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:14:23.004132 10605 ts_tablet_manager.cc:1397] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.003s
Aug 09 16:14:22 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292062, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:14:23.005620 10605 raft_consensus.cc:348] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:23.005946 10605 raft_consensus.cc:729] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545ebbee9c8a4a7a8e7b39352dce92b9, State: Initialized, Role: FOLLOWER
I20260809 16:14:23.006114 10605 consensus_queue.cc:260] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } }
I20260809 16:14:23.006415 10605 ts_tablet_manager.cc:1428] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:14:23.011035 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:22.995140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:59767)
0809 16:14:22.995411 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:22.995456 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:22.996071 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:22.996703 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:22.996707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:22.996989 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:22.997174 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:22.997179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:22.998860 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:22.998861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.000082 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.000085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.000991 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.000992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.001128 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.003701 (+ 2573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:23.003712 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:23.005441 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:23.009069 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:23.009077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:23.009079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:23.009355 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:23.009688 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:23.009689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:23.009690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:23.009830 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:23.010353 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:23.010361 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:23.010540 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.010653 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.010830 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":93,"threads_started":1}
I20260809 16:14:23.011709 10780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:22.995220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:59767 (local address 127.16.224.190:35019)
0809 16:14:22.996280 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:22.996290 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:22.996330 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:14:22.996465 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:22.996471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:22.996527 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:22.996659 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:22.997297 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:22.998717 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.000179 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.000772 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.000872 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.005565 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:23.005570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:23.005570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:23.005583 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:23.008972 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.009477 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.009481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.009481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.009610 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.009905 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.009906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.009907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.010219 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:23.010299 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.011353 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.011516 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1006,"thread_start_us":78,"threads_started":1}
I20260809 16:14:23.012363 10777 heartbeater.cc:347] Connected to a master server at 127.16.224.190:35019
I20260809 16:14:23.012471 10777 heartbeater.cc:464] Registering TS with master...
I20260809 16:14:23.012665 10777 heartbeater.cc:511] Master 127.16.224.190:35019 requested a full tablet report, sending...
I20260809 16:14:23.013463 10470 ts_manager.cc:194] Registered new tserver with Master: 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129:36107)
I20260809 16:14:23.014683 10470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:59767
I20260809 16:14:23.015830 10777 heartbeater.cc:503] Master 127.16.224.190:35019 was elected leader, sending a full tablet report...
I20260809 16:14:23.023134 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.014121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:36107 (local address 127.0.0.1:35058)
0809 16:14:23.015017 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.015055 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:23.015139 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:23.019852 (+ 4713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:23.019855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:23.019858 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:23.019917 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.019919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.020868 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.020869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.021872 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.021876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.022824 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.022826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.022913 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.022929 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.023004 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.023058 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":83,"threads_started":1}
I20260809 16:14:23.023383 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.014425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.16.224.129:36107)
0809 16:14:23.019648 (+ 5223us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.019653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.019663 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:14:23.019695 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.019697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.019711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.019779 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:23.019974 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.020786 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.021967 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.022732 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.022809 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.022861 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.023254 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.023300 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5173,"thread_start_us":87,"threads_started":1}
I20260809 16:14:23.023737 10672 tablet_service.cc:1514] Processing DeleteTablet for tablet 687f71bca96846b2ae24ca1a81f1d310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:14:02 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:35058
I20260809 16:14:23.024327 10790 tablet_replica.cc:330] stopping tablet replica
I20260809 16:14:23.024432 10790 raft_consensus.cc:2229] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:14:23.024505 10790 raft_consensus.cc:2258] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:14:23.024811 10790 ts_tablet_manager.cc:1905] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:14:23.027163 10790 ts_tablet_manager.cc:1918] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:14:23.027240 10790 log.cc:1196] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal/wals/687f71bca96846b2ae24ca1a81f1d310
I20260809 16:14:23.027493 10790 ts_tablet_manager.cc:1939] T 687f71bca96846b2ae24ca1a81f1d310 P 545ebbee9c8a4a7a8e7b39352dce92b9: Deleting consensus metadata
I20260809 16:14:23.028038 10448 catalog_manager.cc:4841] TS 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129:36107): tablet 687f71bca96846b2ae24ca1a81f1d310 (table test-table [id=3ec72bde1d3d4d9d9acaeb5a4b128b7e]) successfully deleted
W20260809 16:14:23.100224 10782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:23.100427 10782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:23.100450 10782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:23.101531 10782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:23.101562 10782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:23.101585 10782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:14:23.102991 10782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:46187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=46335
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:23.103173 10782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:23.103356 10782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:23.108080 10800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:23.108131 10795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:23.108165 10782 server_base.cc:1034] running on GCE node
W20260809 16:14:23.108081 10796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:23.108529 10782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:23.108755 10782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:23.109906 10782 hybrid_clock.cc:648] HybridClock initialized: now 1786292063109885 us; error 33 us; skew 500 ppm
Aug 09 16:14:23 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:23.112347 10782 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:14:23.112737 10782 webserver.cc:460] Webserver started at http://127.16.224.130:46335/ using document root <none> and password file <none>
I20260809 16:14:23.112927 10782 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:23.112991 10782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:23.114336 10782 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:14:23.115073 10811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:23.115283 10782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:14:23.115379 10782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "462e502be3d845da872817ceb9f54ef6"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "1720b409fe437776b76cb7f5a1c62bc3"
server_key_iv: "039af7ac391d1c5ee5fdd2a6cb51b175"
server_key_version: "encryptionkey@0"
I20260809 16:14:23.115724 10782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:23.131235 10782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:23.132959 10782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:23.133072 10782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:23.133281 10782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:23.133697 10825 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260809 16:14:23.135206 10825 ts_tablet_manager.cc:1852] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 16:14:23.135497 10825 ts_tablet_manager.cc:1905] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:14:23.137564 10825 ts_tablet_manager.cc:1918] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:14:23.137645 10825 log.cc:1196] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/687f71bca96846b2ae24ca1a81f1d310
I20260809 16:14:23.137881 10825 ts_tablet_manager.cc:1939] T 687f71bca96846b2ae24ca1a81f1d310 P 462e502be3d845da872817ceb9f54ef6: Deleting consensus metadata
I20260809 16:14:23.138082 10782 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 16:14:23.138127 10782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260809 16:14:23.138165 10782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:14:23.138190 10782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:23.143986 10782 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:46187
I20260809 16:14:23.144063 10985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:46187 every 8 connection(s)
I20260809 16:14:23.144325 10782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:14:23 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:14:23.152329 10780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.145027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:33269 (local address 127.16.224.190:35019)
0809 16:14:23.145109 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.145111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.145295 (+ 184us) server_negotiation.cc:407] Connection header received
0809 16:14:23.145760 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.145761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.145770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.145822 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:23.146422 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.147002 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.147933 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.148139 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.148192 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.150147 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:23.150149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:23.150149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:23.150156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:23.151298 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.151650 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.151651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.151651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.151697 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.151868 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.151869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.151869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.151964 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:23.152009 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.152201 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.152264 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:14:23.152482 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.145012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:33269)
0809 16:14:23.145178 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.145215 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:23.145683 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:23.145886 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:23.145889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:23.146137 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:23.146353 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.146359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.147069 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.147069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.147855 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.147857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.148199 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.148200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.148290 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.148756 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:23.148764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:23.150064 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:23.151372 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:23.151379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:23.151380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:23.151597 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:23.151740 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:23.151740 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:23.151741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:23.151827 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:23.152004 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:23.152007 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:23.152111 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.152183 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.152295 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260809 16:14:23.152927 10986 heartbeater.cc:347] Connected to a master server at 127.16.224.190:35019
I20260809 16:14:23.153023 10986 heartbeater.cc:464] Registering TS with master...
I20260809 16:14:23.153190 10986 heartbeater.cc:511] Master 127.16.224.190:35019 requested a full tablet report, sending...
I20260809 16:14:23.153529 10470 ts_manager.cc:194] Registered new tserver with Master: 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
I20260809 16:14:23.153864 10470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:33269
I20260809 16:14:23.154027 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 10782
I20260809 16:14:23.154302 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:44465
--local_ip_for_outbound_sockets=127.16.224.131
--tserver_master_addrs=127.16.224.190:35019
--webserver_port=33747
--webserver_interface=127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:23.236271 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:23.236488 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:23.236510 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:23.237612 10991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:23.237648 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:23.237671 10991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:14:23.239145 10991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:44465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=33747
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:23.239339 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:23.239526 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:23.244323 11001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:23.244433 11002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:23.244375 11004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:23.244524 10991 server_base.cc:1034] running on GCE node
I20260809 16:14:23.244753 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:23.245018 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:23.246178 10991 hybrid_clock.cc:648] HybridClock initialized: now 1786292063246155 us; error 36 us; skew 500 ppm
Aug 09 16:14:23 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:23.249070 10991 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:14:23.249542 10991 webserver.cc:460] Webserver started at http://127.16.224.131:33747/ using document root <none> and password file <none>
I20260809 16:14:23.249760 10991 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:23.249823 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:23.251200 10991 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:14:23.251796 11015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:23.251956 10991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:23.252041 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "50966eca2d4f4bdd8f34a32812e61849"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "919d59b4ef88f3942a2d580f2c51e1e6"
server_key_iv: "9383995e212ae6230413f2cec90648d2"
server_key_version: "encryptionkey@0"
I20260809 16:14:23.252382 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:23.265568 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:23.267341 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:23.267485 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:23.267689 10991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:23.267972 10991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:14:23.268005 10991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:23.268034 10991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:14:23.268059 10991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:23.273633 10991 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:44465
I20260809 16:14:23.273694 11187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:44465 every 8 connection(s)
I20260809 16:14:23.274013 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:14:23.279681 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 10991
Aug 09 16:14:23 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:14:23.283195 10780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.274758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:50875 (local address 127.16.224.190:35019)
0809 16:14:23.274823 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.274825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.275022 (+ 197us) server_negotiation.cc:407] Connection header received
0809 16:14:23.275528 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.275530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.275544 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.275596 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:23.276538 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.277026 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.278021 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.278211 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.278266 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.280410 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:23.280411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:23.280412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:23.280419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:23.281858 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.282290 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.282290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.282291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.282367 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.282630 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.282630 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.282631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.282726 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:23.282779 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.283043 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.283124 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:14:23.283407 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.274725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:50875)
0809 16:14:23.274906 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.274942 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:23.275431 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:23.275668 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:23.275672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:23.276249 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:23.276443 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.276448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.277110 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.277111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.277927 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.277929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.278279 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.278280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.278434 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.278856 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:23.278872 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:23.280307 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:23.281925 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:23.281932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:23.281934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:23.282221 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:23.282438 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:23.282438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:23.282439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:23.282572 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:23.282780 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:23.282786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:23.282928 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.283012 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.283175 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260809 16:14:23.283880 11190 heartbeater.cc:347] Connected to a master server at 127.16.224.190:35019
I20260809 16:14:23.283985 11190 heartbeater.cc:464] Registering TS with master...
I20260809 16:14:23.284194 11190 heartbeater.cc:511] Master 127.16.224.190:35019 requested a full tablet report, sending...
I20260809 16:14:23.284638 10470 ts_manager.cc:194] Registered new tserver with Master: 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465)
I20260809 16:14:23.285100 10470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:50875
I20260809 16:14:23.286646 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.280113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.16.224.190:35019)
0809 16:14:23.280234 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.280235 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.280323 (+ 88us) server_negotiation.cc:407] Connection header received
0809 16:14:23.280408 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.280409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.280422 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.280496 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:23.281036 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.281764 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.282781 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.283006 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.283067 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.284442 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:23.284443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:23.284443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:23.284450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:23.285771 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.286073 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.286073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.286074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.286118 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.286237 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.286237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.286238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.286343 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:23.286482 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.286509 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.286571 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
I20260809 16:14:23.287286 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:14:23.289829 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.287614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46206 (local address 127.16.224.190:35019)
0809 16:14:23.287646 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.287648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.287767 (+ 119us) server_negotiation.cc:407] Connection header received
0809 16:14:23.287830 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.287831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.287869 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.287935 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:14:23.288032 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.288481 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.289169 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.289371 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.289424 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.289572 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:14:23.289620 (+ 48us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 16:14:23.289773 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46206: token expired
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 16:14:23.289884 11198 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46206: token expired
W20260809 16:14:23.290730 10366 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.16.224.190:35019)
I20260809 16:14:23.295848 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.290960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.16.224.190:35019)
0809 16:14:23.291003 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.291004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.291011 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:14:23.291030 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.291031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.291039 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.291084 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:23.291508 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.291941 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.292579 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.292753 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.292802 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.293808 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:23.293810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:23.293810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:23.293817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:23.294873 (+ 1056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.295170 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.295171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.295172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.295221 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:23.295377 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:23.295378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:23.295379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:23.295509 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:23.295597 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.295693 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.295762 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:14:23.296968 10470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:14:23.298636 10470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:14:23.300688 10672 tablet_service.cc:1467] Processing CreateTablet for tablet 5e485f747c224db8996c34170796e11d (DEFAULT_TABLE table=test-table [id=a187bae4dfaf4be0830666c344d1e5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:14:23.300901 10672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e485f747c224db8996c34170796e11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:14:23.302598 10605 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap starting.
I20260809 16:14:23.302906 10605 tablet_bootstrap.cc:654] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Neither blocks nor log segments found. Creating new log.
I20260809 16:14:23.303356 10605 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: No bootstrap required, opened a new log
I20260809 16:14:23.303406 10605 ts_tablet_manager.cc:1397] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:14:23.303493 10605 raft_consensus.cc:348] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.303565 10605 raft_consensus.cc:374] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:14:23.303614 10605 raft_consensus.cc:729] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545ebbee9c8a4a7a8e7b39352dce92b9, State: Initialized, Role: FOLLOWER
I20260809 16:14:23.303664 10605 consensus_queue.cc:260] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.303781 10605 ts_tablet_manager.cc:1428] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:23.304917 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.300402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44465 (local address 127.0.0.1:58126)
0809 16:14:23.300533 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.300544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:23.300582 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:23.300885 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:23.300886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:23.300888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:23.300933 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.300935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.301996 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.301997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.303175 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.303179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.304674 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.304675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.304741 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.304746 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.304796 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.304850 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 16:14:23.304944 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.300505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.16.224.131:44465)
0809 16:14:23.300704 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.300709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.300725 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:14:23.300771 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.300773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.300790 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.301002 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:23.301090 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.301924 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.303856 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.304592 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.304674 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.304737 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.304822 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.304867 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":127,"threads_started":1}
I20260809 16:14:23.305672 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.300677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.16.224.130:46187)
0809 16:14:23.302083 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.302087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.302097 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:14:23.302135 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.302136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.302152 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.302230 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:23.302415 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.303454 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.304754 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.305366 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.305429 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.305464 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.305587 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.305614 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1362,"thread_start_us":91,"threads_started":1}
I20260809 16:14:23.305691 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.300439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46187 (local address 127.0.0.1:40484)
0809 16:14:23.300733 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.300743 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:23.300905 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:23.302241 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:23.302242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:23.302244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:23.302287 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.302288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.303536 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.303537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.304664 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.304667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.305445 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.305447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.305509 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.305513 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.305568 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.305609 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":95,"threads_started":1}
I20260809 16:14:23.305725 11092 tablet_service.cc:1467] Processing CreateTablet for tablet 5e485f747c224db8996c34170796e11d (DEFAULT_TABLE table=test-table [id=a187bae4dfaf4be0830666c344d1e5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:14:23.306082 11092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e485f747c224db8996c34170796e11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:14:23.306341 10892 tablet_service.cc:1467] Processing CreateTablet for tablet 5e485f747c224db8996c34170796e11d (DEFAULT_TABLE table=test-table [id=a187bae4dfaf4be0830666c344d1e5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:14:23.306655 10892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e485f747c224db8996c34170796e11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:14:23.308797 10825 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Bootstrap starting.
I20260809 16:14:23.309577 10825 tablet_bootstrap.cc:654] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Neither blocks nor log segments found. Creating new log.
I20260809 16:14:23.309602 11204 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Bootstrap starting.
I20260809 16:14:23.309922 10825 log.cc:824] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:23.310271 11204 tablet_bootstrap.cc:654] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Neither blocks nor log segments found. Creating new log.
I20260809 16:14:23.310745 10825 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: No bootstrap required, opened a new log
I20260809 16:14:23.310808 10825 ts_tablet_manager.cc:1397] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:14:23.310909 11204 log.cc:824] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:23.312736 10825 raft_consensus.cc:348] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.312932 10825 raft_consensus.cc:374] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:14:23.312995 10825 raft_consensus.cc:729] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462e502be3d845da872817ceb9f54ef6, State: Initialized, Role: FOLLOWER
I20260809 16:14:23.313117 10825 consensus_queue.cc:260] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.313442 10825 ts_tablet_manager.cc:1428] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:14:23.313452 10986 heartbeater.cc:503] Master 127.16.224.190:35019 was elected leader, sending a full tablet report...
I20260809 16:14:23.314713 11204 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: No bootstrap required, opened a new log
I20260809 16:14:23.314790 11204 ts_tablet_manager.cc:1397] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 16:14:23.316597 11204 raft_consensus.cc:348] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.316788 11204 raft_consensus.cc:374] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:14:23.316838 11204 raft_consensus.cc:729] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50966eca2d4f4bdd8f34a32812e61849, State: Initialized, Role: FOLLOWER
I20260809 16:14:23.316951 11204 consensus_queue.cc:260] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.317245 11204 ts_tablet_manager.cc:1428] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:14:23.317370 11190 heartbeater.cc:503] Master 127.16.224.190:35019 was elected leader, sending a full tablet report...
I20260809 16:14:23.348184 11209 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:23.348340 11209 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.348788 11209 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187), 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129:36107)
I20260809 16:14:23.351501 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.349063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:60959 (local address 127.16.224.129:36107)
0809 16:14:23.349118 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.349120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.349182 (+ 62us) server_negotiation.cc:407] Connection header received
0809 16:14:23.349240 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.349241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.349251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.349297 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:23.349443 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.349906 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.350836 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.351195 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.351256 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.351266 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.351399 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.351433 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:14:23.351545 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.348938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:36107 (local address 127.16.224.131:60959)
0809 16:14:23.349124 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.349131 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:23.349189 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:23.349306 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:23.349307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:23.349308 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:23.349381 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.349383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.349977 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.349978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.350762 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.350764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.351268 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.351269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.351339 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.351343 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.351387 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.351433 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260809 16:14:23.351567 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.349015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:59059 (local address 127.16.224.130:46187)
0809 16:14:23.349066 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.349068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.349134 (+ 66us) server_negotiation.cc:407] Connection header received
0809 16:14:23.349166 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.349167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.349176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.349238 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:14:23.349399 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.349883 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.350857 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.351232 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.351310 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.351319 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.351464 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.351499 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:14:23.351595 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.348922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46187 (local address 127.16.224.131:59059)
0809 16:14:23.348974 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.348985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:23.349023 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:23.349252 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:23.349254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:23.349256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:14:23.349330 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.349332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.349962 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.349963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.350757 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:23.350759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.351315 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.351315 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:23.351389 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.351392 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:14:23.351439 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:14:23.351479 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":13}
I20260809 16:14:23.351855 10704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e485f747c224db8996c34170796e11d" candidate_uuid: "50966eca2d4f4bdd8f34a32812e61849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" is_pre_election: true
I20260809 16:14:23.351941 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e485f747c224db8996c34170796e11d" candidate_uuid: "50966eca2d4f4bdd8f34a32812e61849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462e502be3d845da872817ceb9f54ef6" is_pre_election: true
I20260809 16:14:23.352007 10704 raft_consensus.cc:2454] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50966eca2d4f4bdd8f34a32812e61849 in term 0.
I20260809 16:14:23.352113 10920 raft_consensus.cc:2454] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50966eca2d4f4bdd8f34a32812e61849 in term 0.
I20260809 16:14:23.352236 11017 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50966eca2d4f4bdd8f34a32812e61849, 545ebbee9c8a4a7a8e7b39352dce92b9; no voters:
I20260809 16:14:23.352350 11209 raft_consensus.cc:2781] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:14:23.352416 11209 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:14:23.352435 11209 raft_consensus.cc:3037] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:14:23.353199 11209 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.353339 11209 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [CANDIDATE]: Term 1 election: Requested vote from peers 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187), 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129:36107)
I20260809 16:14:23.353468 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e485f747c224db8996c34170796e11d" candidate_uuid: "50966eca2d4f4bdd8f34a32812e61849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462e502be3d845da872817ceb9f54ef6"
I20260809 16:14:23.353498 10704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e485f747c224db8996c34170796e11d" candidate_uuid: "50966eca2d4f4bdd8f34a32812e61849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9"
I20260809 16:14:23.353551 10920 raft_consensus.cc:3037] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:14:23.353561 10704 raft_consensus.cc:3037] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:14:23.354233 10920 raft_consensus.cc:2454] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50966eca2d4f4bdd8f34a32812e61849 in term 1.
I20260809 16:14:23.354429 11019 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 462e502be3d845da872817ceb9f54ef6, 50966eca2d4f4bdd8f34a32812e61849; no voters:
I20260809 16:14:23.354516 11209 raft_consensus.cc:2781] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:14:23.354656 11209 raft_consensus.cc:686] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 1 LEADER]: Becoming Leader. State: Replica: 50966eca2d4f4bdd8f34a32812e61849, State: Running, Role: LEADER
I20260809 16:14:23.354759 10704 raft_consensus.cc:2454] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50966eca2d4f4bdd8f34a32812e61849 in term 1.
I20260809 16:14:23.354748 11209 consensus_queue.cc:237] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:23.355634 10469 catalog_manager.cc:5495] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131). New cstate: current_term: 1 leader_uuid: "50966eca2d4f4bdd8f34a32812e61849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260809 16:14:23.364586 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.362539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.16.224.131:44465)
0809 16:14:23.362577 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:23.362579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:23.362590 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:14:23.362656 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:23.362656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:23.362672 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:23.362743 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:14:23.362856 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.363302 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.364011 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:23.364188 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:23.364231 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:23.364344 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:14:23.364439 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:14:23.364491 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:23.364503 (+ 12us) server_negotiation.cc:299] Negotiation successful
0809 16:14:23.364517 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:14:23.366277 10704 raft_consensus.cc:1261] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Refusing update from remote peer 50966eca2d4f4bdd8f34a32812e61849: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:14:23.366276 10920 raft_consensus.cc:1261] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Refusing update from remote peer 50966eca2d4f4bdd8f34a32812e61849: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:14:23.366632 11212 consensus_queue.cc:1035] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [LEADER]: Connected to new peer: Peer: permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:14:23.366765 11209 consensus_queue.cc:1035] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [LEADER]: Connected to new peer: Peer: permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:14:23.369068 11217 mvcc.cc:205] Tried to move back new op lower bound from 7316652291545808896 to 7316652291501719552. Current Snapshot: MvccSnapshot[applied={T|T < 7316652291545808896}]
I20260809 16:14:23.369863 11218 mvcc.cc:205] Tried to move back new op lower bound from 7316652291545808896 to 7316652291501719552. Current Snapshot: MvccSnapshot[applied={T|T < 7316652291545808896}]
I20260809 16:14:23.371240 10469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:14:23.371323 10469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:14:23.372025 10469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Sending DeleteTablet for 3 replicas of tablet 5e485f747c224db8996c34170796e11d
I20260809 16:14:23.372324 11092 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e485f747c224db8996c34170796e11d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:14:23 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:58126
I20260809 16:14:23.372336 10892 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e485f747c224db8996c34170796e11d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:14:23 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:40484
I20260809 16:14:23.372351 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 10534
I20260809 16:14:23.388652 11232 tablet_replica.cc:330] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: stopping tablet replica
I20260809 16:14:23.388779 11232 raft_consensus.cc:2229] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:14:23.388885 11232 raft_consensus.cc:2258] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:14:23.389047 11231 tablet_replica.cc:330] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: stopping tablet replica
I20260809 16:14:23.389142 11231 raft_consensus.cc:2229] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:14:23.389227 11231 raft_consensus.cc:2258] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:14:23.390456 11231 ts_tablet_manager.cc:1905] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:14:23.390450 11232 ts_tablet_manager.cc:1905] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:14:23.394102 11231 ts_tablet_manager.cc:1918] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:14:23.394186 11231 log.cc:1196] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/5e485f747c224db8996c34170796e11d
I20260809 16:14:23.394454 11232 ts_tablet_manager.cc:1918] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:14:23.394531 11231 ts_tablet_manager.cc:1939] T 5e485f747c224db8996c34170796e11d P 50966eca2d4f4bdd8f34a32812e61849: Deleting consensus metadata
I20260809 16:14:23.394634 11232 log.cc:1196] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/5e485f747c224db8996c34170796e11d
I20260809 16:14:23.394929 11232 ts_tablet_manager.cc:1939] T 5e485f747c224db8996c34170796e11d P 462e502be3d845da872817ceb9f54ef6: Deleting consensus metadata
I20260809 16:14:23.395006 10449 catalog_manager.cc:4841] TS 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): tablet 5e485f747c224db8996c34170796e11d (table test-table [id=a187bae4dfaf4be0830666c344d1e5bb]) successfully deleted
I20260809 16:14:23.395476 10450 catalog_manager.cc:4841] TS 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187): tablet 5e485f747c224db8996c34170796e11d (table test-table [id=a187bae4dfaf4be0830666c344d1e5bb]) successfully deleted
W20260809 16:14:23.398485 10448 connection.cc:535] client connection to 127.16.224.129:36107 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:14:23.398607 10448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:14:23.398959 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 10782
I20260809 16:14:23.399106 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:23.398970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:36107 (local address 127.0.0.1:35070)
0809 16:14:23.399029 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:23.399070 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:36107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 16:14:23.406513 10448 catalog_manager.cc:4584] TS 545ebbee9c8a4a7a8e7b39352dce92b9 (127.16.224.129:36107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e485f747c224db8996c34170796e11d: Network error: Client connection negotiation failed: client connection to 127.16.224.129:36107: connect: Connection refused (error 111)
I20260809 16:14:23.411322 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 10991
I20260809 16:14:23.430411 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 10420
I20260809 16:14:43.441807 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:35019
--webserver_interface=127.16.224.190
--webserver_port=43113
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:35019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:43.525060 11234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:43.525276 11234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:43.525298 11234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:43.526410 11234 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:14:43.526444 11234 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:14:43.526455 11234 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:14:43.526465 11234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:14:43.526477 11234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:43.526489 11234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:43.526504 11234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:14:43.526515 11234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:14:43.527940 11234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:35019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:35019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:43.528133 11234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:43.528313 11234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:43.533125 11242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:43.533267 11234 server_base.cc:1034] running on GCE node
W20260809 16:14:43.533124 11246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:43.533208 11243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:43.533545 11234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:43.533836 11234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:43.534987 11234 hybrid_clock.cc:648] HybridClock initialized: now 1786292083534972 us; error 29 us; skew 500 ppm
Aug 09 16:14:43 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:43.537361 11234 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:14:43.537750 11234 webserver.cc:460] Webserver started at http://127.16.224.190:43113/ using document root <none> and password file <none>
I20260809 16:14:43.537961 11234 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:43.538018 11234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:43.539335 11234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:14:43.539955 11255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.540097 11234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:14:43.540182 11234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "907c31399a4b45afb99db75efe84bd0d"
format_stamp: "Formatted at 2026-08-09 16:14:01 on dist-test-slave-fgb3"
server_key: "03e78414f57dfc642738a863bf6a10e3"
server_key_iv: "285f9a62aac474bba83ca7fcfb4ef558"
server_key_version: "encryptionkey@0"
I20260809 16:14:43.540514 11234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:43.555828 11234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:43.557591 11234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:43.557726 11234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:43.561656 11234 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:35019
I20260809 16:14:43.561699 11341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:35019 every 8 connection(s)
I20260809 16:14:43.562045 11234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:14:43.563510 11342 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:14:43.564159 11342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Bootstrap starting.
I20260809 16:14:43.568372 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 11234
I20260809 16:14:43.568660 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:36107
--local_ip_for_outbound_sockets=127.16.224.129
--tserver_master_addrs=127.16.224.190:35019
--webserver_port=40301
--webserver_interface=127.16.224.129
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:14:43.573050 11342 log.cc:824] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:43.575281 11342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:14:43.575579 11342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Bootstrap complete.
I20260809 16:14:43.577682 11342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:43.577905 11342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907c31399a4b45afb99db75efe84bd0d, State: Initialized, Role: FOLLOWER
I20260809 16:14:43.578073 11342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:43.578140 11342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:14:43.578181 11342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:14:43.578226 11342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:14:43.579244 11342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:43.579337 11342 leader_election.cc:302] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907c31399a4b45afb99db75efe84bd0d; no voters:
I20260809 16:14:43.579492 11342 leader_election.cc:288] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:14:43.579543 11347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:14:43.579674 11347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [term 3 LEADER]: Becoming Leader. State: Replica: 907c31399a4b45afb99db75efe84bd0d, State: Running, Role: LEADER
I20260809 16:14:43.579759 11342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:14:43.579763 11347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } }
I20260809 16:14:43.580132 11349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "907c31399a4b45afb99db75efe84bd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } } }
I20260809 16:14:43.580159 11347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907c31399a4b45afb99db75efe84bd0d. Latest consensus state: current_term: 3 leader_uuid: "907c31399a4b45afb99db75efe84bd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907c31399a4b45afb99db75efe84bd0d" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 35019 } } }
I20260809 16:14:43.580287 11349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: This master's current role is: LEADER
I20260809 16:14:43.580304 11347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d [sys.catalog]: This master's current role is: LEADER
I20260809 16:14:43.580627 11361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:14:43.581504 11361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:14:43.581612 11361 catalog_manager.cc:1259] Loaded cluster ID: 7f0a02396b344c4bba3f95a8126821af
I20260809 16:14:43.581643 11361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:14:43.582799 11361 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:14:43.583007 11361 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Loaded TSK: 0
I20260809 16:14:43.583058 11361 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 907c31399a4b45afb99db75efe84bd0d: Loaded TSK: 1
F20260809 16:14:43.583281 11361 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f505f2be9ad google::LogMessage::Fail() at ??:0
@ 0x7f505f2c188a google::LogMessage::SendToLog() at ??:0
@ 0x7f505f2be4de google::LogMessage::Flush() at ??:0
@ 0x7f505f2bfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5061a2b104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5061a28d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f506721c1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f506721c48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f506721c594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f50671e2c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f50671ecad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f506118c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5061181954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5061fe56db start_thread at ??:0
@ 0x7f505ddbc61f clone at ??:0
W20260809 16:14:43.684162 11345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:43.684448 11345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:43.684485 11345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:43.686126 11345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:43.686189 11345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:43.686236 11345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:14:43.688395 11345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:36107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=40301
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:43.688674 11345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:43.688956 11345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:43.694448 11413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:43.694482 11411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:43.694622 11345 server_base.cc:1034] running on GCE node
W20260809 16:14:43.694687 11410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:43.694907 11345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:43.695179 11345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:43.696352 11345 hybrid_clock.cc:648] HybridClock initialized: now 1786292083696338 us; error 26 us; skew 500 ppm
Aug 09 16:14:43 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:43.699462 11345 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:14:43.700024 11345 webserver.cc:460] Webserver started at http://127.16.224.129:40301/ using document root <none> and password file <none>
I20260809 16:14:43.700320 11345 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:43.700410 11345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:43.702142 11345 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:14:43.702946 11426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.703119 11345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:14:43.703212 11345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "545ebbee9c8a4a7a8e7b39352dce92b9"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "4cbde485e3f470198e11b1cf2aeb0623"
server_key_iv: "1c748487f8cdbe7bda2abc2582dce630"
server_key_version: "encryptionkey@0"
I20260809 16:14:43.703579 11345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:43.717514 11345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:43.719810 11345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:43.719944 11345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:43.720175 11345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:43.720645 11439 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:14:43.722699 11345 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:14:43.722744 11345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:14:43.722777 11345 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:14:43.723426 11345 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:14:43.723464 11345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:14:43.723551 11439 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap starting.
I20260809 16:14:43.730253 11345 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:36107
I20260809 16:14:43.730445 11599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:36107 every 8 connection(s)
I20260809 16:14:43.730705 11345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:14:43.734572 11439 log.cc:824] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:14:43.734675 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 11345
I20260809 16:14:43.734958 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:46187
--local_ip_for_outbound_sockets=127.16.224.130
--tserver_master_addrs=127.16.224.190:35019
--webserver_port=46335
--webserver_interface=127.16.224.130
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:14:43.737205 11439 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:14:43.737529 11439 tablet_bootstrap.cc:492] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Bootstrap complete.
I20260809 16:14:43.738663 11439 ts_tablet_manager.cc:1397] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.015s
I20260809 16:14:43.740056 11439 raft_consensus.cc:348] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:43.740402 11439 raft_consensus.cc:729] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545ebbee9c8a4a7a8e7b39352dce92b9, State: Initialized, Role: FOLLOWER
I20260809 16:14:43.740566 11439 consensus_queue.cc:260] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:43.740844 11439 ts_tablet_manager.cc:1428] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260809 16:14:43.796257 11604 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:14:43.731417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:43495)
0809 16:14:43.731587 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.731632 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:43.732091 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:43.796032 (+ 63941us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
W20260809 16:14:43.796523 11600 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:35019 (0 consecutive failures): Network error: Failed to ping master at 127.16.224.190:35019: Client connection negotiation failed: client connection to 127.16.224.190:35019: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:14:43.796895 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.796769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:39669)
0809 16:14:43.796828 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.796871 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:14:43.797199 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.797122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:60937)
0809 16:14:43.797161 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.797183 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:14:43.797329 11600 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:14:43.820089 11607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:43.820303 11607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:43.820326 11607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:43.821408 11607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:43.821440 11607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:43.821465 11607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:14:43.822909 11607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:46187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=46335
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:43.823097 11607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:43.823282 11607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:43.828109 11620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:43.828114 11623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:43.828114 11619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:43.828202 11607 server_base.cc:1034] running on GCE node
I20260809 16:14:43.828505 11607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:43.828714 11607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:43.829845 11607 hybrid_clock.cc:648] HybridClock initialized: now 1786292083829840 us; error 35 us; skew 500 ppm
Aug 09 16:14:43 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:43.832345 11607 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:14:43.832765 11607 webserver.cc:460] Webserver started at http://127.16.224.130:46335/ using document root <none> and password file <none>
I20260809 16:14:43.832950 11607 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:43.832989 11607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:43.834357 11607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:14:43.834944 11632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.835099 11607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.835197 11607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "462e502be3d845da872817ceb9f54ef6"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "1720b409fe437776b76cb7f5a1c62bc3"
server_key_iv: "039af7ac391d1c5ee5fdd2a6cb51b175"
server_key_version: "encryptionkey@0"
I20260809 16:14:43.835561 11607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:43.845300 11607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:43.847268 11607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:43.847405 11607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:43.847609 11607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:43.847888 11607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:14:43.847923 11607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.847954 11607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:14:43.847976 11607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.853581 11607 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:46187
I20260809 16:14:43.853645 11822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:46187 every 8 connection(s)
I20260809 16:14:43.853950 11607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:14:43.854943 11825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.854579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:45397)
0809 16:14:43.854728 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.854795 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
W20260809 16:14:43.855295 11823 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:35019 (0 consecutive failures): Network error: Failed to ping master at 127.16.224.190:35019: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
I20260809 16:14:43.855530 11825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.855449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:33713)
0809 16:14:43.855489 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.855514 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:14:43.855718 11825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.855671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:42463)
0809 16:14:43.855690 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.855707 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 16:14:43.855795 11823 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:14:43.861516 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 11607
I20260809 16:14:43.861763 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:44465
--local_ip_for_outbound_sockets=127.16.224.131
--tserver_master_addrs=127.16.224.190:35019
--webserver_port=33747
--webserver_interface=127.16.224.131
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:14:43.943897 11827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:14:43.944097 11827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:14:43.944119 11827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:14:43.945206 11827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:14:43.945243 11827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:14:43.945267 11827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:14:43.946695 11827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:44465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=33747
--tserver_master_addrs=127.16.224.190:35019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:14:43.946915 11827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:14:43.947111 11827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:14:43.951987 11835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:43.952122 11827 server_base.cc:1034] running on GCE node
W20260809 16:14:43.952201 11839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:14:43.952023 11836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:14:43.952587 11827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:14:43.952852 11827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:14:43.953990 11827 hybrid_clock.cc:648] HybridClock initialized: now 1786292083953976 us; error 29 us; skew 500 ppm
Aug 09 16:14:43 dist-test-slave-fgb3 krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:14:43.956447 11827 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:14:43.956883 11827 webserver.cc:460] Webserver started at http://127.16.224.131:33747/ using document root <none> and password file <none>
I20260809 16:14:43.957094 11827 fs_manager.cc:359] Metadata directory not provided
I20260809 16:14:43.957139 11827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:14:43.958464 11827 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:14:43.959028 11848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.959194 11827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.959280 11827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "50966eca2d4f4bdd8f34a32812e61849"
format_stamp: "Formatted at 2026-08-09 16:14:02 on dist-test-slave-fgb3"
server_key: "919d59b4ef88f3942a2d580f2c51e1e6"
server_key_iv: "9383995e212ae6230413f2cec90648d2"
server_key_version: "encryptionkey@0"
I20260809 16:14:43.959646 11827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:14:43.970104 11827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:14:43.972182 11827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:14:43.972319 11827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:14:43.972534 11827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:14:43.972824 11827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:14:43.972857 11827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.972879 11827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:14:43.972903 11827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:14:43.978772 11827 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:44465
I20260809 16:14:43.978845 12009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:44465 every 8 connection(s)
I20260809 16:14:43.979154 11827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:14:43.980122 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.979765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:43117)
0809 16:14:43.979916 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.979983 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
W20260809 16:14:43.980474 12010 heartbeater.cc:644] Failed to heartbeat to 127.16.224.190:35019 (0 consecutive failures): Network error: Failed to ping master at 127.16.224.190:35019: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
I20260809 16:14:43.980697 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.980629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:39509)
0809 16:14:43.980658 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.980682 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 16:14:43.980974 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:43.980894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:42649)
0809 16:14:43.980932 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:43.980954 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:14:43.981088 12010 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:14:43.987411 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 11827
I20260809 16:14:44.797961 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:44.797713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:34975)
0809 16:14:44.797839 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:44.797885 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260809 16:14:44.856252 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:44.856058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:53267)
0809 16:14:44.856174 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:44.856206 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260809 16:14:44.981546 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:44.981354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:51311)
0809 16:14:44.981465 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:44.981497 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260809 16:14:45.601337 12019 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:45.601480 12019 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:45.601858 12019 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
Aug 09 16:14:45 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.131@KRBTEST.COM
Aug 09 16:14:45 dist-test-slave-fgb3 krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.130@KRBTEST.COM
I20260809 16:14:45.612011 12023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:45.602226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:44299 (local address 127.16.224.131:44465)
0809 16:14:45.602393 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:45.602405 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:45.602429 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:14:45.602857 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:45.602859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:45.602901 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:45.603025 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:45.603816 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.604692 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.605705 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.606164 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.606341 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:45.608632 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:45.608637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:45.608638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:45.608651 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:45.610479 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:45.610901 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:45.610905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:45.610905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:45.610999 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:45.611212 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:45.611213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:45.611213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:45.611442 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:45.611515 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:45.611789 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 16:14:45.611907 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260809 16:14:45.612001 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:45.602033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44465 (local address 127.16.224.129:44299)
0809 16:14:45.602302 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:45.602349 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:45.602390 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:45.603090 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:45.603093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:45.603503 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:45.603692 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:45.603698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.604794 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.604795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:45.605626 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:45.605628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.606285 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.606286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:45.606467 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:45.607060 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:45.607064 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:45.608523 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:45.610556 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:45.610563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:45.610566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:45.610838 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:45.611057 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:45.611058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:45.611058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:45.611156 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:45.611505 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:45.611511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:45.611635 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 16:14:45.611720 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:14:45.611897 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":50,"thread_start_us":80,"threads_started":1}
I20260809 16:14:45.612088 12022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:45.602174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:58813 (local address 127.16.224.130:46187)
0809 16:14:45.602361 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 16:14:45.602375 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 16:14:45.602404 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:14:45.602876 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:14:45.602879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:14:45.602918 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:14:45.603062 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:14:45.603824 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.604681 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.605714 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.606164 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.606689 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:45.608993 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:14:45.608996 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:14:45.608997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:14:45.609006 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:14:45.610733 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:45.611062 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:45.611065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:45.611065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:45.611159 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:14:45.611344 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:14:45.611344 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:14:45.611345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:14:45.611581 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:14:45.611661 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:14:45.611845 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 16:14:45.611977 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":57,"threads_started":1}
I20260809 16:14:45.612104 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:45.602033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:46187 (local address 127.16.224.129:58813)
0809 16:14:45.602159 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:45.602171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:14:45.602226 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:14:45.603091 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:14:45.603095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:14:45.603501 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:14:45.603692 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:45.603698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.604793 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.604794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:45.605627 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:14:45.605630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:14:45.606297 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:14:45.606298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:14:45.606442 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:14:45.607016 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:14:45.607024 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:14:45.608910 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:14:45.610815 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:45.610816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:45.610816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:45.610973 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:45.611225 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:14:45.611225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:14:45.611226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:14:45.611286 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:14:45.611654 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:14:45.611655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:14:45.611702 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:14:45.611768 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:14:45.611980 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":148,"thread_start_us":61,"threads_started":1}
W20260809 16:14:45.612921 11436 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
W20260809 16:14:45.613019 11433 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
I20260809 16:14:45.613070 11433 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 545ebbee9c8a4a7a8e7b39352dce92b9; no voters: 462e502be3d845da872817ceb9f54ef6, 50966eca2d4f4bdd8f34a32812e61849
I20260809 16:14:45.613173 12019 raft_consensus.cc:2726] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:14:45.798671 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:45.798516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:42975)
0809 16:14:45.798584 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:45.798628 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:14:45.856984 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:45.856726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:34019)
0809 16:14:45.856873 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:45.856933 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260809 16:14:45.982167 12027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:45.981941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:42893)
0809 16:14:45.982066 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:45.982112 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260809 16:14:46.799376 12028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:46.799145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:55535)
0809 16:14:46.799270 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:46.799306 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260809 16:14:46.857564 12029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:46.857373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:47221)
0809 16:14:46.857490 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:46.857523 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260809 16:14:46.982748 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:46.982548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:44989)
0809 16:14:46.982666 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:46.982703 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260809 16:14:47.488858 12031 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:47.488958 12031 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:47.489125 12031 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
W20260809 16:14:47.489527 11436 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
W20260809 16:14:47.489621 11433 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
I20260809 16:14:47.489655 11433 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 545ebbee9c8a4a7a8e7b39352dce92b9; no voters: 462e502be3d845da872817ceb9f54ef6, 50966eca2d4f4bdd8f34a32812e61849
I20260809 16:14:47.489739 12031 raft_consensus.cc:2726] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:14:47.800081 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:47.799851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:47399)
0809 16:14:47.799989 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:47.800031 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260809 16:14:47.858150 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:47.857945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:57297)
0809 16:14:47.858068 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:47.858101 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260809 16:14:47.983382 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:47.983158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:55459)
0809 16:14:47.983286 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:47.983324 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260809 16:14:48.800822 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:48.800608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:40589)
0809 16:14:48.800736 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:48.800771 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260809 16:14:48.858712 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:48.858499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:53841)
0809 16:14:48.858625 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:48.858668 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260809 16:14:48.984004 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:48.983767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:49861)
0809 16:14:48.983916 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:48.983957 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260809 16:14:49.801542 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:49.801298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:37231)
0809 16:14:49.801449 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:49.801490 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260809 16:14:49.859341 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:49.859127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:52803)
0809 16:14:49.859254 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:49.859297 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260809 16:14:49.984690 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:49.984480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:49129)
0809 16:14:49.984603 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:49.984645 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260809 16:14:50.800613 12046 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:50.800719 12046 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:50.800897 12046 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
W20260809 16:14:50.801347 11436 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
W20260809 16:14:50.801445 11433 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
I20260809 16:14:50.801487 11433 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 545ebbee9c8a4a7a8e7b39352dce92b9; no voters: 462e502be3d845da872817ceb9f54ef6, 50966eca2d4f4bdd8f34a32812e61849
I20260809 16:14:50.801559 12046 raft_consensus.cc:2726] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:14:50.802117 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:50.801945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:51531)
0809 16:14:50.802044 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:50.802081 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
I20260809 16:14:50.859938 12049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:50.859721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:51791)
0809 16:14:50.859857 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:50.859895 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260809 16:14:50.985369 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:50.985140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:50491)
0809 16:14:50.985282 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:50.985321 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260809 16:14:51.802757 12053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:51.802544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:43023)
0809 16:14:51.802669 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:51.802707 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 16:14:51.860533 12056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:51.860350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:60605)
0809 16:14:51.860458 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:51.860493 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260809 16:14:51.986059 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:51.985826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:40975)
0809 16:14:51.985973 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:51.986013 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260809 16:14:52.537173 12058 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:52.537285 12058 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:52.537470 12058 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
W20260809 16:14:52.537920 11436 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
W20260809 16:14:52.538039 11433 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
I20260809 16:14:52.538084 11433 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 545ebbee9c8a4a7a8e7b39352dce92b9; no voters: 462e502be3d845da872817ceb9f54ef6, 50966eca2d4f4bdd8f34a32812e61849
I20260809 16:14:52.538153 12058 raft_consensus.cc:2726] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:14:52.803417 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:52.803197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:55701)
0809 16:14:52.803324 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:52.803362 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260809 16:14:52.861214 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:52.860971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:41795)
0809 16:14:52.861115 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:52.861159 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260809 16:14:52.986714 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:52.986499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:39195)
0809 16:14:52.986637 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:52.986672 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260809 16:14:53.804138 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:53.803912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:59471)
0809 16:14:53.804046 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:53.804091 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260809 16:14:53.861871 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:53.861677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:60991)
0809 16:14:53.861791 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:53.861826 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260809 16:14:53.987363 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:53.987142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:53069)
0809 16:14:53.987266 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:53.987315 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260809 16:14:54.804908 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:54.804654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:45295)
0809 16:14:54.804809 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:54.804857 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260809 16:14:54.862509 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:54.862264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:34877)
0809 16:14:54.862406 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:54.862459 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260809 16:14:54.988011 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:54.987775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:41439)
0809 16:14:54.987924 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:54.987963 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260809 16:14:55.805624 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:55.805394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:37041)
0809 16:14:55.805531 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:55.805573 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260809 16:14:55.863117 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:55.862921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:51325)
0809 16:14:55.863040 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:55.863073 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260809 16:14:55.914827 12074 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:55.914924 12074 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:55.915098 12074 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
W20260809 16:14:55.915447 11433 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
W20260809 16:14:55.915536 11436 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
I20260809 16:14:55.915580 11436 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 545ebbee9c8a4a7a8e7b39352dce92b9; no voters: 462e502be3d845da872817ceb9f54ef6, 50966eca2d4f4bdd8f34a32812e61849
I20260809 16:14:55.915648 12074 raft_consensus.cc:2726] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:14:55.988587 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:55.988396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:34309)
0809 16:14:55.988513 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:55.988545 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260809 16:14:56.806371 12076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:56.806121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:46041)
0809 16:14:56.806247 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:56.806287 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260809 16:14:56.863781 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:56.863574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:60563)
0809 16:14:56.863694 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:56.863734 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260809 16:14:56.989285 12081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:56.989043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:40597)
0809 16:14:56.989183 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:56.989231 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260809 16:14:57.773901 12084 raft_consensus.cc:482] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:14:57.774019 12084 raft_consensus.cc:504] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50966eca2d4f4bdd8f34a32812e61849" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44465 } } peers { permanent_uuid: "462e502be3d845da872817ceb9f54ef6" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 46187 } } peers { permanent_uuid: "545ebbee9c8a4a7a8e7b39352dce92b9" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 36107 } }
I20260809 16:14:57.774194 12084 leader_election.cc:288] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465), 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187)
W20260809 16:14:57.774657 11436 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 462e502be3d845da872817ceb9f54ef6 (127.16.224.130:46187): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
W20260809 16:14:57.774755 11433 leader_election.cc:341] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50966eca2d4f4bdd8f34a32812e61849 (127.16.224.131:44465): Not found: Tablet not found: 5e485f747c224db8996c34170796e11d
I20260809 16:14:57.774828 11433 leader_election.cc:302] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 545ebbee9c8a4a7a8e7b39352dce92b9; no voters: 462e502be3d845da872817ceb9f54ef6, 50966eca2d4f4bdd8f34a32812e61849
I20260809 16:14:57.774915 12084 raft_consensus.cc:2726] T 5e485f747c224db8996c34170796e11d P 545ebbee9c8a4a7a8e7b39352dce92b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:14:57.807029 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:57.806790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:44959)
0809 16:14:57.806940 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:57.806981 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260809 16:14:57.864452 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:57.864210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:40257)
0809 16:14:57.864344 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:57.864380 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260809 16:14:57.989989 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:57.989747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:53829)
0809 16:14:57.989878 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:57.989916 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260809 16:14:58.807799 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:58.807535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.129:40071)
0809 16:14:58.807706 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:58.807742 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260809 16:14:58.865091 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:58.864894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.130:52481)
0809 16:14:58.865021 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:58.865051 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260809 16:14:58.990736 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:14:58.990520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:35019 (local address 127.16.224.131:52353)
0809 16:14:58.990649 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:14:58.990687 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.190:35019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260809 16:14:58.993860 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 11345
I20260809 16:14:59.029500 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 11607
I20260809 16:14:59.043532 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 11827
2026-08-09T16:14:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58486 ms)
[----------] 30 tests from SecurityITest (230546 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:14:59 dist-test-slave-fgb3 krb5kdc[12095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:14:59 dist-test-slave-fgb3 krb5kdc[12095](info): set up 2 sockets
Aug 09 16:14:59 dist-test-slave-fgb3 krb5kdc[12095](info): commencing operation
krb5kdc: starting...
W20260809 16:15:00.209061 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:00.326699 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:37749
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:00.408595 12115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:00.408797 12115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:00.408818 12115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:00.409876 12115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:00.409907 12115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:00.409920 12115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:00.409932 12115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:00.409942 12115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:00.411427 12115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:37749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:37749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:00.411633 12115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:00.411808 12115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:00.416673 12122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:00.416730 12123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:00.416862 12127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:00.416687 12115 server_base.cc:1034] running on GCE node
I20260809 16:15:00.417176 12115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:00.417452 12115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:00.418591 12115 hybrid_clock.cc:648] HybridClock initialized: now 1786292100418581 us; error 31 us; skew 500 ppm
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:00.420928 12115 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:00.421350 12115 webserver.cc:460] Webserver started at http://127.16.224.190:36121/ using document root <none> and password file <none>
I20260809 16:15:00.421533 12115 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:00.421573 12115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:00.421664 12115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:00.422729 12115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "0387cac203f7476b848af1bf5f8f2b7a"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "f9920f818762cb882a166d37d3ec432c"
server_key_iv: "79ccd6ec60b3e2b98b5ba3250d043060"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.423034 12115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "0387cac203f7476b848af1bf5f8f2b7a"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "f9920f818762cb882a166d37d3ec432c"
server_key_iv: "79ccd6ec60b3e2b98b5ba3250d043060"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.424227 12115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:00.424934 12137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.425137 12115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:15:00.425233 12115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "0387cac203f7476b848af1bf5f8f2b7a"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "f9920f818762cb882a166d37d3ec432c"
server_key_iv: "79ccd6ec60b3e2b98b5ba3250d043060"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.425307 12115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:00.441512 12115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:00.443819 12115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:00.443943 12115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:00.448081 12115 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:37749
I20260809 16:15:00.448129 12223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:37749 every 8 connection(s)
I20260809 16:15:00.448477 12115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:00.449357 12224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:00.451433 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 12115
I20260809 16:15:00.451535 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:00.451738 17282 external_mini_cluster.cc:1428] Setting key d3b825abad48e1a2003c471df9c66906
I20260809 16:15:00.451771 12224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a: Bootstrap starting.
I20260809 16:15:00.452472 12224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:00.452802 12224 log.cc:824] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:00.453581 12224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a: No bootstrap required, opened a new log
I20260809 16:15:00.455440 12224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0387cac203f7476b848af1bf5f8f2b7a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37749 } }
I20260809 16:15:00.455632 12224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:00.455668 12224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0387cac203f7476b848af1bf5f8f2b7a, State: Initialized, Role: FOLLOWER
I20260809 16:15:00.455801 12224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0387cac203f7476b848af1bf5f8f2b7a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37749 } }
I20260809 16:15:00.455870 12224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:00.455905 12224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:00.455946 12224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:00.456509 12224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0387cac203f7476b848af1bf5f8f2b7a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37749 } }
I20260809 16:15:00.456604 12224 leader_election.cc:302] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0387cac203f7476b848af1bf5f8f2b7a; no voters:
I20260809 16:15:00.456758 12224 leader_election.cc:288] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:00.456799 12232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:00.456977 12232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [term 1 LEADER]: Becoming Leader. State: Replica: 0387cac203f7476b848af1bf5f8f2b7a, State: Running, Role: LEADER
I20260809 16:15:00.457041 12224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:00.457690 12232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0387cac203f7476b848af1bf5f8f2b7a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37749 } }
I20260809 16:15:00.458353 12233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0387cac203f7476b848af1bf5f8f2b7a. Latest consensus state: current_term: 1 leader_uuid: "0387cac203f7476b848af1bf5f8f2b7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0387cac203f7476b848af1bf5f8f2b7a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37749 } } }
I20260809 16:15:00.459095 12233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:00.458582 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0387cac203f7476b848af1bf5f8f2b7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0387cac203f7476b848af1bf5f8f2b7a" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37749 } } }
W20260809 16:15:00.458875 12256 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 16:15:00.459336 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a [sys.catalog]: This master's current role is: LEADER
W20260809 16:15:00.459372 12256 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:00.459491 12263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:00.459672 12263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:00.461361 12263 catalog_manager.cc:1347] Generated new cluster ID: 515ac610155d413b96c7be0042a8b7e4
I20260809 16:15:00.461412 12263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:00.464725 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:00.452327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.16.224.190:37749)
0809 16:15:00.452546 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:00.452557 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:00.452596 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:15:00.453169 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:00.453177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:00.453229 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:00.453379 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:00.454038 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.456057 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.456953 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.457473 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.457584 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:00.460641 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:00.460644 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:00.460645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:00.460655 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:00.462604 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:00.463043 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:00.463047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:00.463047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:00.463163 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:00.463395 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:00.463396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:00.463396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:00.463707 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:00.463883 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:00.464363 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 16:15:00.464511 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
I20260809 16:15:00.498194 12263 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:00.498798 12263 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:00.509569 12263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a: Generated new TSK 0
I20260809 16:15:00.509754 12263 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:15:00.544611 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37749
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:00.648471 12268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:00.648691 12268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:00.648715 12268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:00.649845 12268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:00.649883 12268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:00.649909 12268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:15:00.651393 12268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:00.651607 12268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:00.651801 12268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:00.656738 12277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:00.656752 12280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:00.656787 12275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:00.656847 12268 server_base.cc:1034] running on GCE node
I20260809 16:15:00.657222 12268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:00.657516 12268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:00.658674 12268 hybrid_clock.cc:648] HybridClock initialized: now 1786292100658657 us; error 30 us; skew 500 ppm
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:00.661142 12268 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:15:00.661576 12268 webserver.cc:460] Webserver started at http://127.16.224.129:40295/ using document root <none> and password file <none>
I20260809 16:15:00.661804 12268 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:00.661854 12268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:00.661983 12268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:00.663069 12268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "e6f6c64a981e4f5d86be38420b9f0a47"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "407a7f46f219790a2fa72071f510a24d"
server_key_iv: "4b77fe4f784dc5fdeab847d24b436707"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.663408 12268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "e6f6c64a981e4f5d86be38420b9f0a47"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "407a7f46f219790a2fa72071f510a24d"
server_key_iv: "4b77fe4f784dc5fdeab847d24b436707"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.664652 12268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:00.665356 12299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.665514 12268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:00.665602 12268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "e6f6c64a981e4f5d86be38420b9f0a47"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "407a7f46f219790a2fa72071f510a24d"
server_key_iv: "4b77fe4f784dc5fdeab847d24b436707"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.665663 12268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:00.678604 12268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:00.680789 12268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:00.680938 12268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:00.681176 12268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:00.681514 12268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:00.681550 12268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.681582 12268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:00.681599 12268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.687507 12268 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:46875
I20260809 16:15:00.687556 12462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:46875 every 8 connection(s)
I20260809 16:15:00.687891 12268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:15:00.690600 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 12268
I20260809 16:15:00.690721 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:15:00.690948 17282 external_mini_cluster.cc:1428] Setting key 6a50556cd8335320058d0a5bdf3a8867
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:00.697506 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:00.688611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:51087 (local address 127.16.224.190:37749)
0809 16:15:00.688679 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:00.688681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:00.688886 (+ 205us) server_negotiation.cc:407] Connection header received
0809 16:15:00.689405 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:00.689407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:00.689418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:00.689480 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:00.690538 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.691280 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.692314 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.692539 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.692597 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:00.694717 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:00.694719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:00.694719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:00.694728 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:00.696202 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:00.696574 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:00.696575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:00.696575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:00.696637 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:00.696855 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:00.696856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:00.696856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:00.696980 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:00.697036 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:00.697306 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 16:15:00.697412 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:15:00.697672 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:00.688582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:37749 (local address 127.16.224.129:51087)
0809 16:15:00.688758 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:00.688798 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:00.689319 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:00.689554 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:00.689558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:00.690151 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:00.690435 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:00.690441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.691368 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.691369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:00.692227 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:00.692231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.692615 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.692616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:00.692742 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:00.693242 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:00.693251 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:00.694626 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:00.696270 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:00.696279 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:00.696282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:00.696507 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:00.696691 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:00.696692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:00.696693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:00.696799 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:00.697034 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:00.697043 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:00.697186 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 16:15:00.697277 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:15:00.697445 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260809 16:15:00.698199 12463 heartbeater.cc:347] Connected to a master server at 127.16.224.190:37749
I20260809 16:15:00.698293 12463 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:00.698506 12463 heartbeater.cc:511] Master 127.16.224.190:37749 requested a full tablet report, sending...
I20260809 16:15:00.699096 12162 ts_manager.cc:194] Registered new tserver with Master: e6f6c64a981e4f5d86be38420b9f0a47 (127.16.224.129:46875)
I20260809 16:15:00.699748 12162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:51087
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:15:00.717132 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37749
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:00.800962 12470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:00.801172 12470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:00.801203 12470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:00.802428 12470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:00.802479 12470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:00.802513 12470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:15:00.804054 12470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:00.804322 12470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:00.804538 12470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:00.809468 12477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:00.809450 12480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:00.809598 12470 server_base.cc:1034] running on GCE node
W20260809 16:15:00.809451 12478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:00.809886 12470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:00.810097 12470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:00.811234 12470 hybrid_clock.cc:648] HybridClock initialized: now 1786292100811223 us; error 26 us; skew 500 ppm
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:00.813674 12470 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:15:00.814069 12470 webserver.cc:460] Webserver started at http://127.16.224.130:39841/ using document root <none> and password file <none>
I20260809 16:15:00.814277 12470 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:00.814352 12470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:00.814455 12470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:00.815471 12470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "99bd432fd5144d298cb3623f61016b10"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "0f22fcc7fb36ddfa6ec9ba5775c877aa"
server_key_iv: "91ab6481251456bf7bb306261d501200"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.815793 12470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "99bd432fd5144d298cb3623f61016b10"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "0f22fcc7fb36ddfa6ec9ba5775c877aa"
server_key_iv: "91ab6481251456bf7bb306261d501200"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.817052 12470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:00.817757 12490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.817929 12470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:00.818040 12470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "99bd432fd5144d298cb3623f61016b10"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "0f22fcc7fb36ddfa6ec9ba5775c877aa"
server_key_iv: "91ab6481251456bf7bb306261d501200"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.818099 12470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:00.839493 12470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:00.841500 12470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:00.841624 12470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:00.841831 12470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:00.842155 12470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:00.842191 12470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.842216 12470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:00.842238 12470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.847697 12470 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:33957
I20260809 16:15:00.847776 12660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:33957 every 8 connection(s)
I20260809 16:15:00.848057 12470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:15:00.852983 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 12470
I20260809 16:15:00.853086 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:15:00.853309 17282 external_mini_cluster.cc:1428] Setting key 2508d6edd11cf7d044e3907d5fe25d80
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:00.857276 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:00.848779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:44785 (local address 127.16.224.190:37749)
0809 16:15:00.848844 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:00.848846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:00.849025 (+ 179us) server_negotiation.cc:407] Connection header received
0809 16:15:00.849514 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:00.849515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:00.849525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:00.849577 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:00.850545 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.851039 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.852019 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.852207 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.852255 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:00.854523 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:00.854525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:00.854526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:00.854533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:00.855935 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:00.856308 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:00.856309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:00.856310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:00.856383 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:00.856619 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:00.856619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:00.856620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:00.856757 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:00.856822 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:00.857137 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 16:15:00.857207 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:15:00.857539 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:00.848733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:37749 (local address 127.16.224.130:44785)
0809 16:15:00.848900 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:00.848944 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:00.849432 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:00.849639 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:00.849643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:00.850205 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:00.850468 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:00.850475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.851110 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.851111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:00.851942 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:00.851944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.852264 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.852266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:00.852359 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:00.852878 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:00.852887 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:00.854432 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:00.856016 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:00.856023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:00.856026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:00.856241 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:00.856441 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:00.856442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:00.856442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:00.856560 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:00.856823 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:00.856829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:00.857006 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 16:15:00.857114 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:15:00.857273 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260809 16:15:00.858047 12661 heartbeater.cc:347] Connected to a master server at 127.16.224.190:37749
I20260809 16:15:00.858145 12661 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:00.858350 12661 heartbeater.cc:511] Master 127.16.224.190:37749 requested a full tablet report, sending...
I20260809 16:15:00.858814 12162 ts_manager.cc:194] Registered new tserver with Master: 99bd432fd5144d298cb3623f61016b10 (127.16.224.130:33957)
I20260809 16:15:00.859187 12162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:44785
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:15:00.879367 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37749
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:00.961338 12666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:00.961541 12666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:00.961563 12666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:00.962710 12666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:00.962750 12666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:00.962779 12666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:15:00.964214 12666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:00.964450 12666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:00.964668 12666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:00.969533 12680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:00.969512 12688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:00.969504 12683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:00.969736 12666 server_base.cc:1034] running on GCE node
I20260809 16:15:00.969977 12666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:00.970173 12666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:00.971309 12666 hybrid_clock.cc:648] HybridClock initialized: now 1786292100971295 us; error 30 us; skew 500 ppm
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:00.973706 12666 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:15:00.974113 12666 webserver.cc:460] Webserver started at http://127.16.224.131:41803/ using document root <none> and password file <none>
I20260809 16:15:00.974350 12666 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:00.974399 12666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:00.974493 12666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:00.975474 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "8d7e03b6966b4dac91a95bdcf93ff544"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "8e23e8ed34f815d6e42b568f5cb298ed"
server_key_iv: "f338d1b4390a49ac263512dc41bbf594"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.975764 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "8d7e03b6966b4dac91a95bdcf93ff544"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "8e23e8ed34f815d6e42b568f5cb298ed"
server_key_iv: "f338d1b4390a49ac263512dc41bbf594"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.976933 12666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:00.977525 12699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.977663 12666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:00.977749 12666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "8d7e03b6966b4dac91a95bdcf93ff544"
format_stamp: "Formatted at 2026-08-09 16:15:00 on dist-test-slave-fgb3"
server_key: "8e23e8ed34f815d6e42b568f5cb298ed"
server_key_iv: "f338d1b4390a49ac263512dc41bbf594"
server_key_version: "encryptionkey@0"
I20260809 16:15:00.977807 12666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:00.985834 12666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:00.987818 12666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:00.987948 12666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:00.988137 12666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:00.988440 12666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:00.988476 12666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.988502 12666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:00.988525 12666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:00.994009 12666 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:46009
I20260809 16:15:00.994046 12876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:46009 every 8 connection(s)
I20260809 16:15:00.994349 12666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:15:00 dist-test-slave-fgb3 krb5kdc[12095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292100, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:01.002638 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:00.994997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:39433 (local address 127.16.224.190:37749)
0809 16:15:00.995038 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:00.995040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:00.995242 (+ 202us) server_negotiation.cc:407] Connection header received
0809 16:15:00.995696 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:00.995698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:00.995707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:00.995761 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:00.996629 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.997114 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.998031 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.998201 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.998239 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.000188 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:01.000190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:01.000190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:01.000197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:01.001507 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:01.001886 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:01.001887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:01.001888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:01.001943 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:01.002120 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:01.002121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:01.002121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:01.002218 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:01.002261 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.002509 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.002572 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:01.002799 12879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:00.994968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:37749 (local address 127.16.224.131:39433)
0809 16:15:00.995123 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:00.995163 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:00.995622 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:00.995814 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:00.995817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:00.996372 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:00.996564 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:00.996569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.997175 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.997175 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:00.997953 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:00.997955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:00.998250 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:00.998251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:00.998378 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:00.998770 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:00.998778 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:01.000042 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:01.001582 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:01.001587 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:01.001589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:01.001834 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:01.001997 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:01.001997 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:01.001998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:01.002086 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:01.002257 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:01.002260 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:01.002408 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.002485 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.002601 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260809 16:15:01.003235 12877 heartbeater.cc:347] Connected to a master server at 127.16.224.190:37749
I20260809 16:15:01.003324 12877 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:01.003494 12877 heartbeater.cc:511] Master 127.16.224.190:37749 requested a full tablet report, sending...
I20260809 16:15:01.003855 12162 ts_manager.cc:194] Registered new tserver with Master: 8d7e03b6966b4dac91a95bdcf93ff544 (127.16.224.131:46009)
I20260809 16:15:01.004218 12162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:39433
I20260809 16:15:01.004447 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 12666
I20260809 16:15:01.004546 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:15:01.004719 17282 external_mini_cluster.cc:1428] Setting key a409c2c71ed23ffcce017ca57698b2c7
I20260809 16:15:01.005451 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:15:01.012588 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.006750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41180 (local address 127.16.224.190:37749)
0809 16:15:01.006815 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.006817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.006915 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:15:01.006970 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.006971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.006981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.007023 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:01.007551 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.008017 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.008721 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.008918 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.008971 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.010243 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:01.010244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:01.010244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:01.010251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:01.011402 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:01.011758 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:01.011759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:01.011759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:01.011818 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:01.012032 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:01.012033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:01.012033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:01.012186 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:01.012272 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.012402 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.012494 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:15:01.014057 12162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:15:01.015762 12162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:15:01.022137 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.018286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:46009 (local address 127.0.0.1:42772)
0809 16:15:01.018434 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.018442 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.018479 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.018800 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.018803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.018805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.018907 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.018910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.019838 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.019839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.020913 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.020916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.021783 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.021784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.021866 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.021883 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.021994 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.022060 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260809 16:15:01.022173 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.018453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.16.224.131:46009)
0809 16:15:01.018646 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.018650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.018658 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:01.018691 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.018693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.018705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.018773 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.018989 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.019738 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.021006 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.021713 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.021774 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.021835 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.022019 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.022060 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
I20260809 16:15:01.022413 12898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.018075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:46875 (local address 127.0.0.1:34982)
0809 16:15:01.018249 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.018289 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.018425 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.018953 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.018953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.018955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.018985 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.018986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.020365 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.020366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.021296 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.021298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.022145 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.022146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.022211 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.022216 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.022245 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.022284 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":35,"thread_start_us":68,"threads_started":1}
I20260809 16:15:01.022706 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.018075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:33957 (local address 127.0.0.1:54740)
0809 16:15:01.018428 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.018439 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.018497 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.018908 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.018909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.018912 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.018972 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.018975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.020356 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.020357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.021532 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.021535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.022513 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.022514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.022574 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.022578 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.022607 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.022642 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":43,"thread_start_us":69,"threads_started":1}
I20260809 16:15:01.022802 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.018178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.16.224.129:46875)
0809 16:15:01.018796 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.018799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.018808 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:15:01.018846 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.018847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.018861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.019061 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.019141 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.020244 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.021369 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.022045 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.022594 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.022663 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.022688 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.022724 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":105,"threads_started":1}
I20260809 16:15:01.023083 12772 tablet_service.cc:1467] Processing CreateTablet for tablet 3e357a93e0b445e588f2ea4eef96a361 (DEFAULT_TABLE table=test-table [id=8bb49937d2864c2091ffccc0bf935264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:01.023259 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.018259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.16.224.130:33957)
0809 16:15:01.018725 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.018730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.018742 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:15:01.018783 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.018785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.018801 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.019061 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.019136 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.020231 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.021616 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.022417 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.023040 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.023104 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.023132 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.023174 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":379,"threads_started":1}
I20260809 16:15:01.023499 12772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e357a93e0b445e588f2ea4eef96a361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:01.023502 12367 tablet_service.cc:1467] Processing CreateTablet for tablet 3e357a93e0b445e588f2ea4eef96a361 (DEFAULT_TABLE table=test-table [id=8bb49937d2864c2091ffccc0bf935264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:01.023813 12367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e357a93e0b445e588f2ea4eef96a361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:01.023799 12563 tablet_service.cc:1467] Processing CreateTablet for tablet 3e357a93e0b445e588f2ea4eef96a361 (DEFAULT_TABLE table=test-table [id=8bb49937d2864c2091ffccc0bf935264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:01.024044 12563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e357a93e0b445e588f2ea4eef96a361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:01.026803 12907 tablet_bootstrap.cc:492] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Bootstrap starting.
I20260809 16:15:01.027045 12909 tablet_bootstrap.cc:492] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Bootstrap starting.
I20260809 16:15:01.027486 12907 tablet_bootstrap.cc:654] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:01.027594 12909 tablet_bootstrap.cc:654] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:01.027992 12907 log.cc:824] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:01.028023 12909 log.cc:824] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:01.028347 12908 tablet_bootstrap.cc:492] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47: Bootstrap starting.
I20260809 16:15:01.028702 12907 tablet_bootstrap.cc:492] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: No bootstrap required, opened a new log
I20260809 16:15:01.028764 12907 ts_tablet_manager.cc:1397] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:01.028872 12908 tablet_bootstrap.cc:654] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:01.029330 12908 log.cc:824] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:01.029881 12909 tablet_bootstrap.cc:492] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: No bootstrap required, opened a new log
I20260809 16:15:01.029956 12909 ts_tablet_manager.cc:1397] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:15:01.030179 12908 tablet_bootstrap.cc:492] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47: No bootstrap required, opened a new log
I20260809 16:15:01.030249 12908 ts_tablet_manager.cc:1397] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:01.030637 12907 raft_consensus.cc:348] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.030805 12907 raft_consensus.cc:374] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:01.030839 12907 raft_consensus.cc:729] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99bd432fd5144d298cb3623f61016b10, State: Initialized, Role: FOLLOWER
I20260809 16:15:01.030990 12907 consensus_queue.cc:260] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.031268 12907 ts_tablet_manager.cc:1428] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:15:01.031411 12661 heartbeater.cc:503] Master 127.16.224.190:37749 was elected leader, sending a full tablet report...
I20260809 16:15:01.032199 12908 raft_consensus.cc:348] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.032384 12908 raft_consensus.cc:374] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:01.032335 12909 raft_consensus.cc:348] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.032462 12908 raft_consensus.cc:729] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f6c64a981e4f5d86be38420b9f0a47, State: Initialized, Role: FOLLOWER
I20260809 16:15:01.032493 12909 raft_consensus.cc:374] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:01.032562 12909 raft_consensus.cc:729] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7e03b6966b4dac91a95bdcf93ff544, State: Initialized, Role: FOLLOWER
I20260809 16:15:01.032605 12908 consensus_queue.cc:260] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.032701 12909 consensus_queue.cc:260] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.032922 12908 ts_tablet_manager.cc:1428] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:15:01.032950 12909 ts_tablet_manager.cc:1428] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:15:01.032974 12877 heartbeater.cc:503] Master 127.16.224.190:37749 was elected leader, sending a full tablet report...
I20260809 16:15:01.032955 12463 heartbeater.cc:503] Master 127.16.224.190:37749 was elected leader, sending a full tablet report...
W20260809 16:15:01.098649 12662 tablet.cc:2369] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:15:01.188616 12464 tablet.cc:2369] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:15:01.244880 12878 tablet.cc:2369] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:15:01.298465 12914 raft_consensus.cc:482] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:15:01.298599 12914 raft_consensus.cc:504] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.299007 12914 leader_election.cc:288] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f6c64a981e4f5d86be38420b9f0a47 (127.16.224.129:46875), 8d7e03b6966b4dac91a95bdcf93ff544 (127.16.224.131:46009)
I20260809 16:15:01.302968 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.299226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:46009 (local address 127.16.224.130:43001)
0809 16:15:01.299285 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.299296 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.299340 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.299499 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.299500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.299502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.299573 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.299574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.300506 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.300508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.301882 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.301886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.302665 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.302666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.302756 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.302758 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.302818 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.302869 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":86}
I20260809 16:15:01.302974 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.299238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:46875 (local address 127.16.224.130:43229)
0809 16:15:01.299428 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.299437 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.299492 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.299605 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.299605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.299607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.299667 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.299668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.300506 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.300507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.301882 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.301886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.302665 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.302666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.302754 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.302758 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.302818 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.302869 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":78,"thread_start_us":119,"threads_started":1}
I20260809 16:15:01.302986 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.299300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:43001 (local address 127.16.224.131:46009)
0809 16:15:01.299349 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.299351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.299365 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:15:01.299399 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.299400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.299410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.299465 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.299641 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.300415 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.301989 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.302589 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.302663 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.302672 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.302861 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.302908 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:15:01.302986 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.299310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:43229 (local address 127.16.224.129:46875)
0809 16:15:01.299358 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.299360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.299503 (+ 143us) server_negotiation.cc:407] Connection header received
0809 16:15:01.299546 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.299547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.299555 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.299589 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.299747 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.300422 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.301989 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.302589 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.302663 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.302672 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.302863 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.302907 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:15:01.303366 12401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e357a93e0b445e588f2ea4eef96a361" candidate_uuid: "99bd432fd5144d298cb3623f61016b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" is_pre_election: true
I20260809 16:15:01.303345 12807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e357a93e0b445e588f2ea4eef96a361" candidate_uuid: "99bd432fd5144d298cb3623f61016b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" is_pre_election: true
I20260809 16:15:01.303514 12807 raft_consensus.cc:2454] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99bd432fd5144d298cb3623f61016b10 in term 0.
I20260809 16:15:01.303514 12401 raft_consensus.cc:2454] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99bd432fd5144d298cb3623f61016b10 in term 0.
I20260809 16:15:01.303772 12498 leader_election.cc:302] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8d7e03b6966b4dac91a95bdcf93ff544, 99bd432fd5144d298cb3623f61016b10, e6f6c64a981e4f5d86be38420b9f0a47; no voters:
I20260809 16:15:01.303913 12914 raft_consensus.cc:2781] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:15:01.303985 12914 raft_consensus.cc:482] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:15:01.304001 12914 raft_consensus.cc:3037] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:01.304706 12914 raft_consensus.cc:504] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.304847 12914 leader_election.cc:288] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [CANDIDATE]: Term 1 election: Requested vote from peers e6f6c64a981e4f5d86be38420b9f0a47 (127.16.224.129:46875), 8d7e03b6966b4dac91a95bdcf93ff544 (127.16.224.131:46009)
I20260809 16:15:01.304996 12401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e357a93e0b445e588f2ea4eef96a361" candidate_uuid: "99bd432fd5144d298cb3623f61016b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f6c64a981e4f5d86be38420b9f0a47"
I20260809 16:15:01.305009 12807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e357a93e0b445e588f2ea4eef96a361" candidate_uuid: "99bd432fd5144d298cb3623f61016b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7e03b6966b4dac91a95bdcf93ff544"
I20260809 16:15:01.305075 12401 raft_consensus.cc:3037] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:01.305075 12807 raft_consensus.cc:3037] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:01.305779 12807 raft_consensus.cc:2454] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99bd432fd5144d298cb3623f61016b10 in term 1.
I20260809 16:15:01.305791 12401 raft_consensus.cc:2454] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99bd432fd5144d298cb3623f61016b10 in term 1.
I20260809 16:15:01.305972 12497 leader_election.cc:302] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99bd432fd5144d298cb3623f61016b10, e6f6c64a981e4f5d86be38420b9f0a47; no voters:
I20260809 16:15:01.306068 12914 raft_consensus.cc:2781] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:01.306218 12914 raft_consensus.cc:686] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 1 LEADER]: Becoming Leader. State: Replica: 99bd432fd5144d298cb3623f61016b10, State: Running, Role: LEADER
I20260809 16:15:01.306304 12914 consensus_queue.cc:237] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } }
I20260809 16:15:01.307188 12161 catalog_manager.cc:5495] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99bd432fd5144d298cb3623f61016b10 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "99bd432fd5144d298cb3623f61016b10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99bd432fd5144d298cb3623f61016b10" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 } health_report { overall_health: UNKNOWN } } }
I20260809 16:15:01.374015 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.371938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.16.224.130:33957)
0809 16:15:01.372002 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.372004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.372012 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:01.372053 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.372054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.372072 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.372130 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:01.372240 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.372703 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.373402 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.373580 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.373620 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.373721 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:01.373821 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:01.373877 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.373892 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.373909 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:15:01.375824 12807 raft_consensus.cc:1261] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 1 FOLLOWER]: Refusing update from remote peer 99bd432fd5144d298cb3623f61016b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:01.375834 12401 raft_consensus.cc:1261] T 3e357a93e0b445e588f2ea4eef96a361 P e6f6c64a981e4f5d86be38420b9f0a47 [term 1 FOLLOWER]: Refusing update from remote peer 99bd432fd5144d298cb3623f61016b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:01.376190 12922 consensus_queue.cc:1035] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d7e03b6966b4dac91a95bdcf93ff544" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 46009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:01.376320 12914 consensus_queue.cc:1035] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f6c64a981e4f5d86be38420b9f0a47" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:01.379050 12927 mvcc.cc:205] Tried to move back new op lower bound from 7316652447232688128 to 7316652446951354368. Current Snapshot: MvccSnapshot[applied={T|T < 7316652447232688128}]
I20260809 16:15:01.379226 12925 mvcc.cc:205] Tried to move back new op lower bound from 7316652447232688128 to 7316652446951354368. Current Snapshot: MvccSnapshot[applied={T|T < 7316652447232688128 or (T in {7316652447232688128})}]
I20260809 16:15:01.379336 12926 mvcc.cc:205] Tried to move back new op lower bound from 7316652447232688128 to 7316652446951354368. Current Snapshot: MvccSnapshot[applied={T|T < 7316652447232688128}]
I20260809 16:15:01.381057 12161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:01.381163 12161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:01.381892 12161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0387cac203f7476b848af1bf5f8f2b7a: Sending DeleteTablet for 3 replicas of tablet 3e357a93e0b445e588f2ea4eef96a361
I20260809 16:15:01.382220 12367 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e357a93e0b445e588f2ea4eef96a361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:01 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:34982
I20260809 16:15:01.382227 12772 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e357a93e0b445e588f2ea4eef96a361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:01 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:42772
I20260809 16:15:01.382476 12941 tablet_replica.cc:330] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: stopping tablet replica
I20260809 16:15:01.382513 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 12268
I20260809 16:15:01.382571 12941 raft_consensus.cc:2229] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:01.382695 12941 raft_consensus.cc:2258] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:01.383508 12941 ts_tablet_manager.cc:1905] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:01.385391 12941 ts_tablet_manager.cc:1918] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:01.385484 12941 log.cc:1196] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/3e357a93e0b445e588f2ea4eef96a361
I20260809 16:15:01.385845 12941 ts_tablet_manager.cc:1939] T 3e357a93e0b445e588f2ea4eef96a361 P 8d7e03b6966b4dac91a95bdcf93ff544: Deleting consensus metadata
I20260809 16:15:01.386379 12144 catalog_manager.cc:4841] TS 8d7e03b6966b4dac91a95bdcf93ff544 (127.16.224.131:46009): tablet 3e357a93e0b445e588f2ea4eef96a361 (table test-table [id=8bb49937d2864c2091ffccc0bf935264]) successfully deleted
I20260809 16:15:01.391675 12563 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e357a93e0b445e588f2ea4eef96a361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:01 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:54740
I20260809 16:15:01.395743 12943 tablet_replica.cc:330] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: stopping tablet replica
I20260809 16:15:01.395897 12943 raft_consensus.cc:2229] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:15:01.396020 12943 raft_consensus.cc:2258] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:01.396529 12943 ts_tablet_manager.cc:1905] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:01.398212 12943 ts_tablet_manager.cc:1918] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:01.398296 12943 log.cc:1196] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/3e357a93e0b445e588f2ea4eef96a361
I20260809 16:15:01.398654 12943 ts_tablet_manager.cc:1939] T 3e357a93e0b445e588f2ea4eef96a361 P 99bd432fd5144d298cb3623f61016b10: Deleting consensus metadata
I20260809 16:15:01.399142 12144 catalog_manager.cc:4841] TS 99bd432fd5144d298cb3623f61016b10 (127.16.224.130:33957): tablet 3e357a93e0b445e588f2ea4eef96a361 (table test-table [id=8bb49937d2864c2091ffccc0bf935264]) successfully deleted
W20260809 16:15:01.400022 12141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:46875 (error 108)
I20260809 16:15:01.400584 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.400432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:46875 (local address 127.0.0.1:34984)
0809 16:15:01.400499 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.400548 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:15:01.400620 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 12470
W20260809 16:15:01.408676 12141 catalog_manager.cc:4584] TS e6f6c64a981e4f5d86be38420b9f0a47 (127.16.224.129:46875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e357a93e0b445e588f2ea4eef96a361: Network error: Client connection negotiation failed: client connection to 127.16.224.129:46875: connect: Connection refused (error 111)
I20260809 16:15:01.414634 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 12666
I20260809 16:15:01.422760 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 12115
2026-08-09T16:15:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2381 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T16:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:01Z Disabled control of system clock
I20260809 16:15:01.457631 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:37523
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:37523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:01.538698 12953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:01.538930 12953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:01.538949 12953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:01.540120 12953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:01.540163 12953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:01.540174 12953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:01.540194 12953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:01.540216 12953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:01.541616 12953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:37523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:37523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:01.541862 12953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:01.542153 12953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:01.546850 12964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:01.546877 12963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:01.546921 12953 server_base.cc:1034] running on GCE node
W20260809 16:15:01.546862 12967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:01.547336 12953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:01.547613 12953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:01.548743 12953 hybrid_clock.cc:648] HybridClock initialized: now 1786292101548729 us; error 32 us; skew 500 ppm
I20260809 16:15:01.550017 12953 webserver.cc:460] Webserver started at http://127.16.224.190:32775/ using document root <none> and password file <none>
I20260809 16:15:01.550237 12953 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:01.550283 12953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:01.550441 12953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:01.551548 12953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "cba27784db68494db7102d0af8628a48"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "9ae9ec7592bbaffe9667a8272be3ec29"
server_key_iv: "4005c0ae9f00a5a83bfd2f8578fa262e"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.551892 12953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "cba27784db68494db7102d0af8628a48"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "9ae9ec7592bbaffe9667a8272be3ec29"
server_key_iv: "4005c0ae9f00a5a83bfd2f8578fa262e"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.553176 12953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:01.553875 12975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.554068 12953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:01.554154 12953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "cba27784db68494db7102d0af8628a48"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "9ae9ec7592bbaffe9667a8272be3ec29"
server_key_iv: "4005c0ae9f00a5a83bfd2f8578fa262e"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.554216 12953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:01.561549 12953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:01.561784 12953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:01.561888 12953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:01.565577 12953 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:37523
I20260809 16:15:01.565629 13048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:37523 every 8 connection(s)
I20260809 16:15:01.565927 12953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:01.566915 13049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:01.569219 13049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48: Bootstrap starting.
I20260809 16:15:01.569808 13049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:01.570098 13049 log.cc:824] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:01.570837 13049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48: No bootstrap required, opened a new log
I20260809 16:15:01.571614 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 12953
I20260809 16:15:01.571712 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:01.571907 17282 external_mini_cluster.cc:1428] Setting key b0c3c65fb89185d4bc4d820d01c9c603
I20260809 16:15:01.572691 13049 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba27784db68494db7102d0af8628a48" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37523 } }
I20260809 16:15:01.572875 13049 raft_consensus.cc:374] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:01.572919 13049 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba27784db68494db7102d0af8628a48, State: Initialized, Role: FOLLOWER
I20260809 16:15:01.573063 13049 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba27784db68494db7102d0af8628a48" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37523 } }
I20260809 16:15:01.573149 13049 raft_consensus.cc:388] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:01.573217 13049 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:01.573266 13049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:01.573937 13049 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba27784db68494db7102d0af8628a48" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37523 } }
I20260809 16:15:01.574056 13049 leader_election.cc:302] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cba27784db68494db7102d0af8628a48; no voters:
I20260809 16:15:01.574213 13049 leader_election.cc:288] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:01.574267 13056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:01.574492 13049 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:01.574507 13056 raft_consensus.cc:686] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [term 1 LEADER]: Becoming Leader. State: Replica: cba27784db68494db7102d0af8628a48, State: Running, Role: LEADER
I20260809 16:15:01.574723 13056 consensus_queue.cc:237] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba27784db68494db7102d0af8628a48" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37523 } }
I20260809 16:15:01.575129 13057 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cba27784db68494db7102d0af8628a48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba27784db68494db7102d0af8628a48" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37523 } } }
I20260809 16:15:01.575206 13057 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:01.575500 13067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:01.576043 13067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:01.577466 13058 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cba27784db68494db7102d0af8628a48. Latest consensus state: current_term: 1 leader_uuid: "cba27784db68494db7102d0af8628a48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba27784db68494db7102d0af8628a48" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37523 } } }
I20260809 16:15:01.577545 13058 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:01.578544 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.572465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54294 (local address 127.16.224.190:37523)
0809 16:15:01.572704 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.572714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.572752 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:15:01.572866 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.572874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.572944 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.573090 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:01.573398 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.575197 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.576421 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.577054 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.577170 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.577810 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:01.577814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:01.577815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:01.578025 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:01.578072 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:01.578088 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:01.578154 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.578216 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.578294 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260809 16:15:01.578867 13067 catalog_manager.cc:1347] Generated new cluster ID: 4730e75e90924a359491a37db42d4999
I20260809 16:15:01.578923 13067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:01.590010 13067 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:01.590801 13067 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:01.596313 13067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48: Generated new TSK 0
I20260809 16:15:01.596511 13067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:01.601918 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37523
--builtin_ntp_servers=127.16.224.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:01.685555 13087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:01.685768 13087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:01.685798 13087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:01.686951 13087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:01.686995 13087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:01.687026 13087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:15:01.688370 13087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:01.688619 13087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:01.688911 13087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:01.693769 13098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:01.693792 13095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:01.694039 13096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:01.694428 13087 server_base.cc:1034] running on GCE node
I20260809 16:15:01.694623 13087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:01.694845 13087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:01.695999 13087 hybrid_clock.cc:648] HybridClock initialized: now 1786292101695984 us; error 32 us; skew 500 ppm
I20260809 16:15:01.697201 13087 webserver.cc:460] Webserver started at http://127.16.224.129:44501/ using document root <none> and password file <none>
I20260809 16:15:01.697431 13087 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:01.697484 13087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:01.697616 13087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:01.698753 13087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "4a6b02c32b2447b9bba72071799858e3"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "caca7999e0de6935cfdd64e67230bbc0"
server_key_iv: "1e053a975beded85615c10eb40b89baf"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.699092 13087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "4a6b02c32b2447b9bba72071799858e3"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "caca7999e0de6935cfdd64e67230bbc0"
server_key_iv: "1e053a975beded85615c10eb40b89baf"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.700325 13087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:01.701037 13108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.701177 13087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:01.701267 13087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "4a6b02c32b2447b9bba72071799858e3"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "caca7999e0de6935cfdd64e67230bbc0"
server_key_iv: "1e053a975beded85615c10eb40b89baf"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.701337 13087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:01.710738 13087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:01.711011 13087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:01.711135 13087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:01.711337 13087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:01.711663 13087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:01.711697 13087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.711719 13087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:01.711731 13087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.717311 13087 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:40395
I20260809 16:15:01.717370 13267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:40395 every 8 connection(s)
I20260809 16:15:01.717684 13087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:15:01.722476 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.718407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:46087 (local address 127.16.224.190:37523)
0809 16:15:01.718469 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.718471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.718696 (+ 225us) server_negotiation.cc:407] Connection header received
0809 16:15:01.719191 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.719193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.719204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.719274 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:01.719907 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.720391 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.721293 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.721474 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.721516 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.722251 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:01.722252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:01.722252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:01.722255 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:01.722265 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:01.722268 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:01.722300 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.722394 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.722418 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:15:01.722666 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.718392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:37523 (local address 127.16.224.129:46087)
0809 16:15:01.718576 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.718618 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.719112 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.719331 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.719335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.719343 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:01.719843 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.719848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.720452 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.720453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.721231 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.721233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.721529 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.721529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.721637 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.722139 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:01.722139 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:01.722168 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:01.722191 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:01.722302 (+ 111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:01.722304 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:01.722306 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.722387 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.722466 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260809 16:15:01.723145 13268 heartbeater.cc:347] Connected to a master server at 127.16.224.190:37523
I20260809 16:15:01.723237 13268 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:01.723412 13268 heartbeater.cc:511] Master 127.16.224.190:37523 requested a full tablet report, sending...
I20260809 16:15:01.723896 12996 ts_manager.cc:194] Registered new tserver with Master: 4a6b02c32b2447b9bba72071799858e3 (127.16.224.129:40395)
I20260809 16:15:01.724462 12996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.129:46087
I20260809 16:15:01.727921 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 13087
I20260809 16:15:01.728005 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:15:01.728163 17282 external_mini_cluster.cc:1428] Setting key e0e053b3caf4431fe5f74ecc581a91ea
I20260809 16:15:01.728981 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37523
--builtin_ntp_servers=127.16.224.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:01.810799 13271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:01.810995 13271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:01.811015 13271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:01.812079 13271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:01.812108 13271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:01.812131 13271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:15:01.813398 13271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:01.813616 13271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:01.813872 13271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:01.818609 13278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:01.818593 13277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:01.818593 13280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:01.818833 13271 server_base.cc:1034] running on GCE node
I20260809 16:15:01.819029 13271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:01.819245 13271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:01.820394 13271 hybrid_clock.cc:648] HybridClock initialized: now 1786292101820381 us; error 29 us; skew 500 ppm
I20260809 16:15:01.821527 13271 webserver.cc:460] Webserver started at http://127.16.224.130:32867/ using document root <none> and password file <none>
I20260809 16:15:01.821758 13271 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:01.821807 13271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:01.821931 13271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:01.822973 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "e840848c1bd545c2979d547f084aa78d"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "8a5a037040e7f01beafd56641098bbf1"
server_key_iv: "25b485f04fa92813e81d2de82b65bdda"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.823308 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "e840848c1bd545c2979d547f084aa78d"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "8a5a037040e7f01beafd56641098bbf1"
server_key_iv: "25b485f04fa92813e81d2de82b65bdda"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.824550 13271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:01.825217 13292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.825359 13271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:01.825444 13271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "e840848c1bd545c2979d547f084aa78d"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "8a5a037040e7f01beafd56641098bbf1"
server_key_iv: "25b485f04fa92813e81d2de82b65bdda"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.825506 13271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:01.845741 13271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:01.846009 13271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:01.846134 13271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:01.846377 13271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:01.846691 13271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:01.846725 13271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.846755 13271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:01.846771 13271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.852196 13271 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:36993
I20260809 16:15:01.852253 13456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:36993 every 8 connection(s)
I20260809 16:15:01.852550 13271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:15:01.854180 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 13271
I20260809 16:15:01.854265 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:15:01.854465 17282 external_mini_cluster.cc:1428] Setting key a070295a6acdda31c0d77c4e3ab291db
I20260809 16:15:01.855409 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37523
--builtin_ntp_servers=127.16.224.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:15:01.857849 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.853201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:48511 (local address 127.16.224.190:37523)
0809 16:15:01.853302 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.853304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.853460 (+ 156us) server_negotiation.cc:407] Connection header received
0809 16:15:01.853937 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.853938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.853950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.854025 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:01.854793 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.855452 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.856496 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.856751 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.856820 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.857593 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:01.857594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:01.857595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:01.857599 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:01.857615 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:01.857620 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:01.857666 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.857739 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.857769 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 16:15:01.858114 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.853157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:37523 (local address 127.16.224.130:48511)
0809 16:15:01.853344 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.853383 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.853847 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.854080 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.854083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.854091 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:01.854712 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.854720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.855615 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.855616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.856429 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.856432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.856824 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.856824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.856936 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.857462 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:01.857462 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:01.857493 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:01.857515 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:01.857672 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:01.857677 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:01.857680 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.857737 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.857837 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260809 16:15:01.858531 13458 heartbeater.cc:347] Connected to a master server at 127.16.224.190:37523
I20260809 16:15:01.858614 13458 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:01.858783 13458 heartbeater.cc:511] Master 127.16.224.190:37523 requested a full tablet report, sending...
I20260809 16:15:01.859122 12996 ts_manager.cc:194] Registered new tserver with Master: e840848c1bd545c2979d547f084aa78d (127.16.224.130:36993)
I20260809 16:15:01.859483 12996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.130:48511
W20260809 16:15:01.935802 13462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:01.936004 13462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:01.936043 13462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:01.937146 13462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:01.937184 13462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:01.937215 13462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:15:01.938578 13462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:37523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:01.938799 13462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:01.939071 13462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:01.943845 13473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:01.943872 13469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:01.943945 13462 server_base.cc:1034] running on GCE node
W20260809 16:15:01.944036 13476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:01.944254 13462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:01.944494 13462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:01.945628 13462 hybrid_clock.cc:648] HybridClock initialized: now 1786292101945616 us; error 31 us; skew 500 ppm
I20260809 16:15:01.946820 13462 webserver.cc:460] Webserver started at http://127.16.224.131:37321/ using document root <none> and password file <none>
I20260809 16:15:01.947041 13462 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:01.947085 13462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:01.947214 13462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:01.948240 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "02cd577f894d49b7e6069837455da854"
server_key_iv: "12d7c4fa2f6ae7d66eb0312f86c58f76"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.948572 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "02cd577f894d49b7e6069837455da854"
server_key_iv: "12d7c4fa2f6ae7d66eb0312f86c58f76"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.949777 13462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:15:01.950434 13492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.950592 13462 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:01.950680 13462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f"
format_stamp: "Formatted at 2026-08-09 16:15:01 on dist-test-slave-fgb3"
server_key: "02cd577f894d49b7e6069837455da854"
server_key_iv: "12d7c4fa2f6ae7d66eb0312f86c58f76"
server_key_version: "encryptionkey@0"
I20260809 16:15:01.950752 13462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:01.959507 13462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:01.959757 13462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:01.959878 13462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:01.960078 13462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:01.960376 13462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:01.960410 13462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.960436 13462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:01.960454 13462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:01.965816 13462 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:44817
I20260809 16:15:01.965886 13664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:44817 every 8 connection(s)
I20260809 16:15:01.966177 13462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:15:01.971012 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.966948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:35473 (local address 127.16.224.190:37523)
0809 16:15:01.967032 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.967035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.967198 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:15:01.967678 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.967680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.967691 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.967740 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:01.968373 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.968872 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.969807 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.969980 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.970023 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.970796 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:01.970797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:01.970798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:01.970802 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:01.970814 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:01.970817 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:01.970848 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.970924 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.970947 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:15:01.971060 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 13462
I20260809 16:15:01.971158 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:15:01.971266 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.966911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:37523 (local address 127.16.224.131:35473)
0809 16:15:01.967085 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.967120 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.967604 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.967798 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.967802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.967811 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:01.968304 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.968309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.968941 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.968942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.969745 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.969748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.970037 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.970038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.970142 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.970675 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:01.970675 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:01.970703 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:01.970727 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:01.970855 (+ 128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:01.970859 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:01.970862 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.970929 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.971042 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260809 16:15:01.971372 17282 external_mini_cluster.cc:1428] Setting key 28e77d55a367639dcc2cb21d6f77827e
I20260809 16:15:01.971813 13665 heartbeater.cc:347] Connected to a master server at 127.16.224.190:37523
I20260809 16:15:01.971922 13665 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:01.972110 13665 heartbeater.cc:511] Master 127.16.224.190:37523 requested a full tablet report, sending...
I20260809 16:15:01.972462 12996 ts_manager.cc:194] Registered new tserver with Master: d0c8d7e3c5ce4aa9b480a3a8d2f8256f (127.16.224.131:44817)
I20260809 16:15:01.972824 12996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.131:35473
I20260809 16:15:01.983000 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:15:01.986989 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.984320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.16.224.190:37523)
0809 16:15:01.984373 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.984375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.984472 (+ 97us) server_negotiation.cc:407] Connection header received
0809 16:15:01.984537 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.984538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.984550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.984591 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:01.984745 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.985211 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.985920 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.986126 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.986180 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.986768 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:01.986769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:01.986770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:01.986774 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:01.986788 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:01.986792 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:01.986836 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.986891 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.986915 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:15:01.988306 12996 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:15:01.989849 12996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:15:01.995862 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.992237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:40395 (local address 127.0.0.1:44882)
0809 16:15:01.992598 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.992607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.992646 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.992814 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.992815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.992816 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.992856 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.992857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.993678 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.993679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.994884 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.994887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.995566 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.995567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.995648 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.995662 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.995733 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.995787 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
I20260809 16:15:01.995873 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.992368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.16.224.129:40395)
0809 16:15:01.992683 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.992688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.992697 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:15:01.992732 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.992734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.992749 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.992816 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.992906 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.993597 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.994972 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.995480 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.995549 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.995648 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.995748 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.995789 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":93,"threads_started":1}
I20260809 16:15:01.996147 13678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.991883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44817 (local address 127.0.0.1:46230)
0809 16:15:01.991995 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.992035 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.992097 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.992464 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.992468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.992471 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.992535 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.992538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.994024 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.994025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.995089 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.995091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.995930 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.995930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.996002 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.996005 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.996030 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.996068 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260809 16:15:01.996238 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.992175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.16.224.130:36993)
0809 16:15:01.992372 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.992376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.992507 (+ 131us) server_negotiation.cc:407] Connection header received
0809 16:15:01.992558 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.992561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.992576 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.992642 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.992858 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.993933 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.995009 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.995736 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.995807 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.995893 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.996122 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.996161 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260809 16:15:01.996400 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.992079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:36993 (local address 127.0.0.1:46310)
0809 16:15:01.992382 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:01.992392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:01.992443 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:01.992714 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:01.992715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:01.992717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:01.992760 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.992761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.994047 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.994048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.994934 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:01.994938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.995928 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.995930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:01.995996 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.996000 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:01.996047 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:15:01.996075 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":12,"thread_start_us":72,"threads_started":1}
I20260809 16:15:01.996477 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:01.991957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46230 (local address 127.16.224.131:44817)
0809 16:15:01.992225 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:01.992231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:01.992242 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:15:01.992284 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:01.992286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:01.992302 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:01.992384 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:01.992762 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.993933 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.995172 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:01.995841 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:01.996245 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:01.996342 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:01.996366 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 16:15:01.996402 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":98,"threads_started":1}
I20260809 16:15:01.996978 13178 tablet_service.cc:1467] Processing CreateTablet for tablet 64b66638d45043929e22dbf69878ac92 (DEFAULT_TABLE table=test-table [id=b19877d95c6f4cf0810efd7664533015]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:01.997114 13566 tablet_service.cc:1467] Processing CreateTablet for tablet 64b66638d45043929e22dbf69878ac92 (DEFAULT_TABLE table=test-table [id=b19877d95c6f4cf0810efd7664533015]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:01.997191 13362 tablet_service.cc:1467] Processing CreateTablet for tablet 64b66638d45043929e22dbf69878ac92 (DEFAULT_TABLE table=test-table [id=b19877d95c6f4cf0810efd7664533015]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:01.997345 13178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b66638d45043929e22dbf69878ac92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:01.997476 13566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b66638d45043929e22dbf69878ac92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:01.997475 13362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b66638d45043929e22dbf69878ac92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:02.000106 13690 tablet_bootstrap.cc:492] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3: Bootstrap starting.
I20260809 16:15:02.000735 13691 tablet_bootstrap.cc:492] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Bootstrap starting.
I20260809 16:15:02.000765 13690 tablet_bootstrap.cc:654] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:02.000960 13692 tablet_bootstrap.cc:492] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Bootstrap starting.
I20260809 16:15:02.001093 13690 log.cc:824] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:02.001603 13692 tablet_bootstrap.cc:654] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:02.001809 13690 tablet_bootstrap.cc:492] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3: No bootstrap required, opened a new log
I20260809 16:15:02.001873 13690 ts_tablet_manager.cc:1397] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:02.001929 13691 tablet_bootstrap.cc:654] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:02.001924 13692 log.cc:824] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:02.002234 13691 log.cc:824] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:02.002717 13692 tablet_bootstrap.cc:492] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: No bootstrap required, opened a new log
I20260809 16:15:02.002791 13692 ts_tablet_manager.cc:1397] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:02.002947 13691 tablet_bootstrap.cc:492] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: No bootstrap required, opened a new log
I20260809 16:15:02.003007 13691 ts_tablet_manager.cc:1397] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:15:02.004395 13690 raft_consensus.cc:348] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.004568 13690 raft_consensus.cc:374] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:02.004603 13690 raft_consensus.cc:729] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6b02c32b2447b9bba72071799858e3, State: Initialized, Role: FOLLOWER
I20260809 16:15:02.004719 13690 consensus_queue.cc:260] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.004976 13690 ts_tablet_manager.cc:1428] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:15:02.005020 13268 heartbeater.cc:503] Master 127.16.224.190:37523 was elected leader, sending a full tablet report...
I20260809 16:15:02.004951 13691 raft_consensus.cc:348] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.005101 13691 raft_consensus.cc:374] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:02.005132 13691 raft_consensus.cc:729] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c8d7e3c5ce4aa9b480a3a8d2f8256f, State: Initialized, Role: FOLLOWER
I20260809 16:15:02.005237 13691 consensus_queue.cc:260] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.005187 13692 raft_consensus.cc:348] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.005317 13692 raft_consensus.cc:374] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:02.005352 13692 raft_consensus.cc:729] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e840848c1bd545c2979d547f084aa78d, State: Initialized, Role: FOLLOWER
I20260809 16:15:02.005503 13691 ts_tablet_manager.cc:1428] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:15:02.005568 13665 heartbeater.cc:503] Master 127.16.224.190:37523 was elected leader, sending a full tablet report...
I20260809 16:15:02.005811 13692 consensus_queue.cc:260] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.006134 13458 heartbeater.cc:503] Master 127.16.224.190:37523 was elected leader, sending a full tablet report...
I20260809 16:15:02.006363 13692 ts_tablet_manager.cc:1428] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 16:15:02.087937 13702 raft_consensus.cc:482] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:15:02.088078 13702 raft_consensus.cc:504] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.088476 13702 leader_election.cc:288] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0c8d7e3c5ce4aa9b480a3a8d2f8256f (127.16.224.131:44817), 4a6b02c32b2447b9bba72071799858e3 (127.16.224.129:40395)
I20260809 16:15:02.091153 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:02.088657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:44817 (local address 127.16.224.130:36545)
0809 16:15:02.088705 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:02.088718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:02.088781 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:02.088923 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:02.088924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:02.088925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:02.088993 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:02.088995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.089587 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.089588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:02.090349 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:02.090351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.090918 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.090919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:02.090992 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:02.090995 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:02.091039 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:15:02.091081 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 16:15:02.091153 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:02.088736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:36545 (local address 127.16.224.131:44817)
0809 16:15:02.088789 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:02.088792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:02.088803 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:15:02.088853 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:02.088854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:02.088863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:02.088909 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:02.089056 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.089518 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.090439 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.090846 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.090908 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:02.090917 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:02.091050 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:15:02.091085 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:15:02.091384 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:02.088878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:40855 (local address 127.16.224.129:40395)
0809 16:15:02.088927 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:02.088928 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:02.088993 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:15:02.089056 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:02.089057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:02.089069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:02.089121 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:02.089266 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.089752 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.090696 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.091040 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.091101 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:02.091108 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:02.091273 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:15:02.091319 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:15:02.091463 13599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b66638d45043929e22dbf69878ac92" candidate_uuid: "e840848c1bd545c2979d547f084aa78d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" is_pre_election: true
I20260809 16:15:02.091583 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:02.088764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:40395 (local address 127.16.224.130:40855)
0809 16:15:02.088943 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:02.088952 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:02.089006 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:02.089128 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:02.089129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:02.089130 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:02.089200 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:02.089202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.089830 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.089831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:02.090610 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:02.090612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.091113 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.091114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:02.091185 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:02.091188 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:02.091466 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 16:15:02.091507 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
I20260809 16:15:02.091645 13599 raft_consensus.cc:2454] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e840848c1bd545c2979d547f084aa78d in term 0.
I20260809 16:15:02.091899 13208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b66638d45043929e22dbf69878ac92" candidate_uuid: "e840848c1bd545c2979d547f084aa78d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a6b02c32b2447b9bba72071799858e3" is_pre_election: true
I20260809 16:15:02.092024 13208 raft_consensus.cc:2454] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e840848c1bd545c2979d547f084aa78d in term 0.
I20260809 16:15:02.092199 13295 leader_election.cc:302] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0c8d7e3c5ce4aa9b480a3a8d2f8256f, e840848c1bd545c2979d547f084aa78d; no voters:
I20260809 16:15:02.092324 13702 raft_consensus.cc:2781] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:15:02.092386 13702 raft_consensus.cc:482] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:15:02.092407 13702 raft_consensus.cc:3037] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:02.093108 13702 raft_consensus.cc:504] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.093227 13702 leader_election.cc:288] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [CANDIDATE]: Term 1 election: Requested vote from peers d0c8d7e3c5ce4aa9b480a3a8d2f8256f (127.16.224.131:44817), 4a6b02c32b2447b9bba72071799858e3 (127.16.224.129:40395)
I20260809 16:15:02.093400 13599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b66638d45043929e22dbf69878ac92" candidate_uuid: "e840848c1bd545c2979d547f084aa78d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f"
I20260809 16:15:02.093420 13208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b66638d45043929e22dbf69878ac92" candidate_uuid: "e840848c1bd545c2979d547f084aa78d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a6b02c32b2447b9bba72071799858e3"
I20260809 16:15:02.093492 13599 raft_consensus.cc:3037] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:02.093492 13208 raft_consensus.cc:3037] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:02.094414 13208 raft_consensus.cc:2454] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e840848c1bd545c2979d547f084aa78d in term 1.
I20260809 16:15:02.094414 13599 raft_consensus.cc:2454] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e840848c1bd545c2979d547f084aa78d in term 1.
I20260809 16:15:02.094579 13295 leader_election.cc:302] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0c8d7e3c5ce4aa9b480a3a8d2f8256f, e840848c1bd545c2979d547f084aa78d; no voters:
I20260809 16:15:02.094655 13702 raft_consensus.cc:2781] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:02.094813 13702 raft_consensus.cc:686] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 1 LEADER]: Becoming Leader. State: Replica: e840848c1bd545c2979d547f084aa78d, State: Running, Role: LEADER
I20260809 16:15:02.094900 13702 consensus_queue.cc:237] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } }
I20260809 16:15:02.095767 12994 catalog_manager.cc:5495] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d reported cstate change: term changed from 0 to 1, leader changed from <none> to e840848c1bd545c2979d547f084aa78d (127.16.224.130). New cstate: current_term: 1 leader_uuid: "e840848c1bd545c2979d547f084aa78d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e840848c1bd545c2979d547f084aa78d" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 } health_report { overall_health: UNKNOWN } } }
W20260809 16:15:02.103022 13460 tablet.cc:2369] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:15:02.110062 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:02.108030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.16.224.130:36993)
0809 16:15:02.108084 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:02.108086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:02.108093 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:02.108123 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:02.108124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:02.108140 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:02.108200 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:02.108358 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.108812 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.109500 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:02.109675 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:02.109717 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:02.109813 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:02.109909 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:02.109964 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:02.109976 (+ 12us) server_negotiation.cc:299] Negotiation successful
0809 16:15:02.109990 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:15:02.111845 13208 raft_consensus.cc:1261] T 64b66638d45043929e22dbf69878ac92 P 4a6b02c32b2447b9bba72071799858e3 [term 1 FOLLOWER]: Refusing update from remote peer e840848c1bd545c2979d547f084aa78d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:02.111845 13599 raft_consensus.cc:1261] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 1 FOLLOWER]: Refusing update from remote peer e840848c1bd545c2979d547f084aa78d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:02.112221 13705 consensus_queue.cc:1035] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a6b02c32b2447b9bba72071799858e3" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 40395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:02.112349 13702 consensus_queue.cc:1035] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0c8d7e3c5ce4aa9b480a3a8d2f8256f" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 44817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:02.114773 13710 mvcc.cc:205] Tried to move back new op lower bound from 7316652450247335936 to 7316652450181378048. Current Snapshot: MvccSnapshot[applied={T|T < 7316652450247335936}]
I20260809 16:15:02.114967 13712 mvcc.cc:205] Tried to move back new op lower bound from 7316652450247335936 to 7316652450181378048. Current Snapshot: MvccSnapshot[applied={T|T < 7316652450247335936}]
I20260809 16:15:02.115214 13711 mvcc.cc:205] Tried to move back new op lower bound from 7316652450247335936 to 7316652450181378048. Current Snapshot: MvccSnapshot[applied={T|T < 7316652450247335936}]
I20260809 16:15:02.117456 12994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:02.117556 12994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:02.118364 12994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cba27784db68494db7102d0af8628a48: Sending DeleteTablet for 3 replicas of tablet 64b66638d45043929e22dbf69878ac92
I20260809 16:15:02.118713 13362 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b66638d45043929e22dbf69878ac92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:02 UTC) from {username='slave'} at 127.0.0.1:46310
I20260809 16:15:02.118757 13178 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b66638d45043929e22dbf69878ac92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:02 UTC) from {username='slave'} at 127.0.0.1:44882
I20260809 16:15:02.118911 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 13087
I20260809 16:15:02.118999 13566 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b66638d45043929e22dbf69878ac92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:02 UTC) from {username='slave'} at 127.0.0.1:46230
I20260809 16:15:02.130257 13724 tablet_replica.cc:330] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: stopping tablet replica
I20260809 16:15:02.130435 13724 raft_consensus.cc:2229] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:02.130582 13724 raft_consensus.cc:2258] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:02.132206 13724 ts_tablet_manager.cc:1905] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:02.133643 13722 tablet_replica.cc:330] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: stopping tablet replica
I20260809 16:15:02.133816 13722 raft_consensus.cc:2229] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:15:02.133845 13724 ts_tablet_manager.cc:1918] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:02.134021 13724 log.cc:1196] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/64b66638d45043929e22dbf69878ac92
I20260809 16:15:02.134022 13722 raft_consensus.cc:2258] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:02.134395 13724 ts_tablet_manager.cc:1939] T 64b66638d45043929e22dbf69878ac92 P d0c8d7e3c5ce4aa9b480a3a8d2f8256f: Deleting consensus metadata
I20260809 16:15:02.134892 12978 catalog_manager.cc:4841] TS d0c8d7e3c5ce4aa9b480a3a8d2f8256f (127.16.224.131:44817): tablet 64b66638d45043929e22dbf69878ac92 (table test-table [id=b19877d95c6f4cf0810efd7664533015]) successfully deleted
I20260809 16:15:02.136193 13722 ts_tablet_manager.cc:1905] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:02.137413 13722 ts_tablet_manager.cc:1918] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:02.137488 13722 log.cc:1196] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/64b66638d45043929e22dbf69878ac92
I20260809 16:15:02.137713 13722 ts_tablet_manager.cc:1939] T 64b66638d45043929e22dbf69878ac92 P e840848c1bd545c2979d547f084aa78d: Deleting consensus metadata
I20260809 16:15:02.138127 12978 catalog_manager.cc:4841] TS e840848c1bd545c2979d547f084aa78d (127.16.224.130:36993): tablet 64b66638d45043929e22dbf69878ac92 (table test-table [id=b19877d95c6f4cf0810efd7664533015]) successfully deleted
W20260809 16:15:02.148779 12978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:40395 (error 108)
I20260809 16:15:02.148976 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 13271
I20260809 16:15:02.164263 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:02.164094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:40395 (local address 127.0.0.1:44894)
0809 16:15:02.164157 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:02.164221 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:40395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:15:02.164453 12978 catalog_manager.cc:4584] TS 4a6b02c32b2447b9bba72071799858e3 (127.16.224.129:40395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64b66638d45043929e22dbf69878ac92: Network error: Client connection negotiation failed: client connection to 127.16.224.129:40395: connect: Connection refused (error 111)
I20260809 16:15:02.168357 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 13462
I20260809 16:15:02.177716 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 12953
2026-08-09T16:15:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (754 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:02 dist-test-slave-fgb3 krb5kdc[13731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:02 dist-test-slave-fgb3 krb5kdc[13731](info): set up 2 sockets
Aug 09 16:15:02 dist-test-slave-fgb3 krb5kdc[13731](info): commencing operation
krb5kdc: starting...
W20260809 16:15:03.356911 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:03 dist-test-slave-fgb3 krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:03.474664 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46427
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:46427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:03.556129 13753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:03.556342 13753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:03.556375 13753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:03.557485 13753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:03.557526 13753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:03.557544 13753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:03.557559 13753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:03.557575 13753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:03.559118 13753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:46427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:03.559369 13753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:03.559576 13753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:15:03.564297 13753 server_base.cc:1034] running on GCE node
W20260809 16:15:03.564280 13762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:03.564271 13763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:03.564280 13766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:03.564729 13753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:03.565006 13753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:03.566140 13753 hybrid_clock.cc:648] HybridClock initialized: now 1786292103566127 us; error 30 us; skew 500 ppm
Aug 09 16:15:03 dist-test-slave-fgb3 krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292103, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:03.568550 13753 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:03.568953 13753 webserver.cc:460] Webserver started at http://127.16.224.190:45471/ using document root <none> and password file <none>
I20260809 16:15:03.569144 13753 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:03.569185 13753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:03.569275 13753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:03.570281 13753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "41baa2a9099c4a889d5229bd6ae434d9"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "44168dc823eba5176ded8d08751bffab"
server_key_iv: "90e58e7ee3247b5bf0c84c95b51aba70"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.570659 13753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "41baa2a9099c4a889d5229bd6ae434d9"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "44168dc823eba5176ded8d08751bffab"
server_key_iv: "90e58e7ee3247b5bf0c84c95b51aba70"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.571853 13753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:15:03.572489 13779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:03.572649 13753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:03.572736 13753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "41baa2a9099c4a889d5229bd6ae434d9"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "44168dc823eba5176ded8d08751bffab"
server_key_iv: "90e58e7ee3247b5bf0c84c95b51aba70"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.572793 13753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:03.589188 13753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:03.591104 13753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:03.591226 13753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:03.595080 13753 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:46427
I20260809 16:15:03.595134 13859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:46427 every 8 connection(s)
I20260809 16:15:03.595450 13753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:03.596306 13860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:03.598675 13860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9: Bootstrap starting.
I20260809 16:15:03.599238 13860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:03.599499 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 13753
I20260809 16:15:03.599512 13860 log.cc:824] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:03.599603 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:03.599789 17282 external_mini_cluster.cc:1428] Setting key 6e3ca7e209c18f3d47c7a7225f31d581
I20260809 16:15:03.600234 13860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9: No bootstrap required, opened a new log
I20260809 16:15:03.602283 13860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41baa2a9099c4a889d5229bd6ae434d9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46427 } }
I20260809 16:15:03.602475 13860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:03.602502 13860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41baa2a9099c4a889d5229bd6ae434d9, State: Initialized, Role: FOLLOWER
I20260809 16:15:03.602623 13860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41baa2a9099c4a889d5229bd6ae434d9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46427 } }
I20260809 16:15:03.602695 13860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:03.602725 13860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:03.602769 13860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:03.603269 13860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41baa2a9099c4a889d5229bd6ae434d9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46427 } }
I20260809 16:15:03.603380 13860 leader_election.cc:302] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41baa2a9099c4a889d5229bd6ae434d9; no voters:
I20260809 16:15:03.603539 13860 leader_election.cc:288] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:03.603569 13866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:03.603681 13866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [term 1 LEADER]: Becoming Leader. State: Replica: 41baa2a9099c4a889d5229bd6ae434d9, State: Running, Role: LEADER
I20260809 16:15:03.603797 13860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:03.603847 13866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41baa2a9099c4a889d5229bd6ae434d9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46427 } }
I20260809 16:15:03.604789 13868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41baa2a9099c4a889d5229bd6ae434d9. Latest consensus state: current_term: 1 leader_uuid: "41baa2a9099c4a889d5229bd6ae434d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41baa2a9099c4a889d5229bd6ae434d9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46427 } } }
I20260809 16:15:03.604887 13868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:03.605144 13867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41baa2a9099c4a889d5229bd6ae434d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41baa2a9099c4a889d5229bd6ae434d9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46427 } } }
I20260809 16:15:03.605227 13867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9 [sys.catalog]: This master's current role is: LEADER
W20260809 16:15:03.605789 13889 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:15:03.605844 13889 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:03.606442 13891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:03.606649 13891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:15:03 dist-test-slave-fgb3 krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:03.608167 13891 catalog_manager.cc:1347] Generated new cluster ID: 334af3ab6c284410bdcecb582cbf107f
I20260809 16:15:03.608217 13891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:03.611908 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:03.600242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.16.224.190:46427)
0809 16:15:03.600437 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:03.600445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:03.600470 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:15:03.601395 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:03.601404 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:03.601452 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:03.601563 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:03.602378 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.603761 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.604798 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.605473 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.605564 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:03.608305 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:03.608309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:03.608309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:03.608321 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:03.610141 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:03.610529 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:03.610532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:03.610532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:03.610626 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:03.610791 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:03.610792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:03.610792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:03.611054 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:03.611181 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:03.611597 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 16:15:03.611721 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260809 16:15:03.639554 13891 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:03.640197 13891 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:03.650452 13891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9: Generated new TSK 0
I20260809 16:15:03.650625 13891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:15:03.681334 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46427
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:03.762998 13898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:03.763200 13898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:03.763222 13898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:03.764302 13898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:03.764336 13898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:03.764360 13898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:15:03.765764 13898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:03.765947 13898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:03.766126 13898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:03.770992 13903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:03.770954 13904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:03.771117 13898 server_base.cc:1034] running on GCE node
W20260809 16:15:03.770939 13908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:03.771397 13898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:03.771641 13898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:03.772784 13898 hybrid_clock.cc:648] HybridClock initialized: now 1786292103772770 us; error 30 us; skew 500 ppm
Aug 09 16:15:03 dist-test-slave-fgb3 krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292103, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:03.775367 13898 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:15:03.775776 13898 webserver.cc:460] Webserver started at http://127.16.224.129:39887/ using document root <none> and password file <none>
I20260809 16:15:03.775972 13898 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:03.776021 13898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:03.776149 13898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:03.777276 13898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "2554249f8dd245889ee61fc2962f168a"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "eaff4411f00e86e297a63fb79278fbe1"
server_key_iv: "014f38cf52022d65abce4997cfb086a0"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.777644 13898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "2554249f8dd245889ee61fc2962f168a"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "eaff4411f00e86e297a63fb79278fbe1"
server_key_iv: "014f38cf52022d65abce4997cfb086a0"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.779072 13898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:03.779800 13918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:03.779951 13898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:03.780037 13898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "2554249f8dd245889ee61fc2962f168a"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "eaff4411f00e86e297a63fb79278fbe1"
server_key_iv: "014f38cf52022d65abce4997cfb086a0"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.780100 13898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:03.804317 13898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:03.806509 13898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:03.806638 13898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:03.806864 13898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:03.807164 13898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:03.807199 13898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:03.807226 13898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:03.807243 13898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:03.812695 13898 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:45303
I20260809 16:15:03.812772 14083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:45303 every 8 connection(s)
I20260809 16:15:03.813036 13898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:15:03.816977 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 13898
I20260809 16:15:03.817088 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:15:03.817277 17282 external_mini_cluster.cc:1428] Setting key c0d56e3bda24acc8bd8c159db852d1cb
Aug 09 16:15:03 dist-test-slave-fgb3 krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292103, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:03.822386 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:03.813705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:58063 (local address 127.16.224.190:46427)
0809 16:15:03.813757 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:03.813759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:03.813954 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:15:03.814547 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:03.814549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:03.814562 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:03.814609 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:03.815550 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.816066 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.817061 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.817315 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.817379 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:03.819759 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:03.819761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:03.819762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:03.819771 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:03.821113 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:03.821489 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:03.821490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:03.821491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:03.821556 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:03.821753 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:03.821753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:03.821754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:03.821885 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:03.821949 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:03.822164 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 16:15:03.822258 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:15:03.822486 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:03.813683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46427 (local address 127.16.224.129:58063)
0809 16:15:03.813842 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:03.813879 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:03.814447 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:03.814693 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:03.814697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:03.815247 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:03.815447 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:03.815452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.816150 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.816152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:03.816967 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:03.816970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.817397 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.817398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:03.817541 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:03.818023 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:03.818033 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:03.819642 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:03.821197 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:03.821203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:03.821205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:03.821421 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:03.821614 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:03.821614 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:03.821615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:03.821703 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:03.821940 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:03.821942 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:03.822052 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 16:15:03.822127 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:15:03.822250 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260809 16:15:03.823022 14084 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46427
I20260809 16:15:03.823117 14084 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:03.823297 14084 heartbeater.cc:511] Master 127.16.224.190:46427 requested a full tablet report, sending...
I20260809 16:15:03.823800 13802 ts_manager.cc:194] Registered new tserver with Master: 2554249f8dd245889ee61fc2962f168a (127.16.224.129:45303)
I20260809 16:15:03.824397 13802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:58063
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:15:03.844666 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46427
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:03.927034 14094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:03.927253 14094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:03.927275 14094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:03.928411 14094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:03.928452 14094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:03.928480 14094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:15:03.929998 14094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:03.930223 14094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:03.930472 14094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:03.935505 14101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:03.935607 14103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:03.935617 14094 server_base.cc:1034] running on GCE node
W20260809 16:15:03.935513 14100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:03.936021 14094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:03.936281 14094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:03.937453 14094 hybrid_clock.cc:648] HybridClock initialized: now 1786292103937430 us; error 43 us; skew 500 ppm
Aug 09 16:15:03 dist-test-slave-fgb3 krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292103, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:03.940024 14094 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:15:03.940471 14094 webserver.cc:460] Webserver started at http://127.16.224.130:35745/ using document root <none> and password file <none>
I20260809 16:15:03.940675 14094 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:03.940719 14094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:03.940855 14094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:03.941953 14094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "5cb05c8e438f4feabd075c2c0f3ce02f"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "2be8839ec2be01347cc79e71c0fcc042"
server_key_iv: "b084a4a95b9ac33ac7570c5d887f31e2"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.942296 14094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "5cb05c8e438f4feabd075c2c0f3ce02f"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "2be8839ec2be01347cc79e71c0fcc042"
server_key_iv: "b084a4a95b9ac33ac7570c5d887f31e2"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.943634 14094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:15:03.944366 14113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:03.944530 14094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:03.944618 14094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "5cb05c8e438f4feabd075c2c0f3ce02f"
format_stamp: "Formatted at 2026-08-09 16:15:03 on dist-test-slave-fgb3"
server_key: "2be8839ec2be01347cc79e71c0fcc042"
server_key_iv: "b084a4a95b9ac33ac7570c5d887f31e2"
server_key_version: "encryptionkey@0"
I20260809 16:15:03.944687 14094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:03.963445 14094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:03.965154 14094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:03.965282 14094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:03.965494 14094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:03.965802 14094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:03.965837 14094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:03.965859 14094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:03.965883 14094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:03.971534 14094 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:35479
I20260809 16:15:03.971588 14272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:35479 every 8 connection(s)
I20260809 16:15:03.971882 14094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:15:03 dist-test-slave-fgb3 krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292103, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:03.980400 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:03.972535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:48811 (local address 127.16.224.190:46427)
0809 16:15:03.972600 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:03.972602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:03.972802 (+ 200us) server_negotiation.cc:407] Connection header received
0809 16:15:03.973266 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:03.973267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:03.973277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:03.973318 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:03.974185 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.974724 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.975707 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.975891 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.975942 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:03.977824 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:03.977825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:03.977826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:03.977832 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:03.979062 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:03.979504 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:03.979506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:03.979506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:03.979569 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:03.979790 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:03.979791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:03.979791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:03.979913 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:03.979971 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:03.980223 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 16:15:03.980303 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:15:03.980427 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 14094
I20260809 16:15:03.980532 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:03.972520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46427 (local address 127.16.224.130:48811)
0809 16:15:03.972684 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:03.972721 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:03.973191 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:03.973374 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:03.973377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:03.973930 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:03.974119 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:03.974124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.974802 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.974803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:03.975646 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:03.975649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:03.975948 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:03.975949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:03.976065 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:03.976485 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:03.976494 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:03.977740 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:03.979144 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:03.979152 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:03.979153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:03.979411 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:03.979627 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:03.979628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:03.979628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:03.979723 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:03.979979 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:03.979985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:03.980108 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 16:15:03.980195 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:15:03.980341 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260809 16:15:03.980551 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:15:03.980772 17282 external_mini_cluster.cc:1428] Setting key 01c2a9b4e8942b1e56edb45bead6ea68
I20260809 16:15:03.981066 14273 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46427
I20260809 16:15:03.981173 14273 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:03.981365 14273 heartbeater.cc:511] Master 127.16.224.190:46427 requested a full tablet report, sending...
I20260809 16:15:03.981757 13802 ts_manager.cc:194] Registered new tserver with Master: 5cb05c8e438f4feabd075c2c0f3ce02f (127.16.224.130:35479)
I20260809 16:15:03.982126 13802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:48811
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:15:04.005091 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46427
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:04.087280 14278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:04.087473 14278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:04.087494 14278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:04.088564 14278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:04.088596 14278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:04.088619 14278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:15:04.090018 14278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:46427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:04.090216 14278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:04.090445 14278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:04.095134 14289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.095161 14291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.095141 14285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:04.095206 14278 server_base.cc:1034] running on GCE node
I20260809 16:15:04.095613 14278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:04.095851 14278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:04.097002 14278 hybrid_clock.cc:648] HybridClock initialized: now 1786292104096982 us; error 29 us; skew 500 ppm
Aug 09 16:15:04 dist-test-slave-fgb3 krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292104, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:04.104724 14278 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:15:04.105162 14278 webserver.cc:460] Webserver started at http://127.16.224.131:43071/ using document root <none> and password file <none>
I20260809 16:15:04.105432 14278 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:04.105497 14278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:04.105654 14278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:04.106760 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "0654a228dd064fa0b78f5ccd10b95722"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "381ca06bfce9fbcf2b529262c1198ed7"
server_key_iv: "ac50710a6ac525ce2aa78ef661691956"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.107060 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "0654a228dd064fa0b78f5ccd10b95722"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "381ca06bfce9fbcf2b529262c1198ed7"
server_key_iv: "ac50710a6ac525ce2aa78ef661691956"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.108229 14278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:04.108848 14304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.108996 14278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:04.109078 14278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "0654a228dd064fa0b78f5ccd10b95722"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "381ca06bfce9fbcf2b529262c1198ed7"
server_key_iv: "ac50710a6ac525ce2aa78ef661691956"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.109148 14278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:04.133268 14278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:04.135483 14278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:04.135620 14278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:04.135845 14278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:04.136152 14278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:04.136188 14278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.136216 14278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:04.136234 14278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.141721 14278 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:45389
I20260809 16:15:04.141762 14479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:45389 every 8 connection(s)
I20260809 16:15:04.142064 14278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:15:04 dist-test-slave-fgb3 krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292104, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:04.150522 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.142776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:44839 (local address 127.16.224.190:46427)
0809 16:15:04.142827 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.142830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.143009 (+ 179us) server_negotiation.cc:407] Connection header received
0809 16:15:04.143479 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.143480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.143490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.143542 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:04.144406 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.144901 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.145867 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.146050 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.146092 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.148131 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:04.148133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:04.148134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:04.148141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:04.149309 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:04.149643 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:04.149643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:04.149644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:04.149686 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:04.149864 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:04.149864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:04.149865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:04.149958 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:04.150013 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.150350 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.150426 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:15:04.150713 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.142739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:46427 (local address 127.16.224.131:44839)
0809 16:15:04.142889 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.142930 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.143406 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.143600 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.143603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.144136 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:04.144334 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.144339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.144981 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.144982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.145797 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.145799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.146106 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.146107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.146208 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.146675 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:04.146683 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:04.148033 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:04.149378 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:04.149384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:04.149386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:04.149587 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:04.149729 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:04.149729 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:04.149730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:04.149824 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:04.150036 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:04.150043 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:04.150173 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.150300 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.150485 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260809 16:15:04.150828 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 14278
I20260809 16:15:04.150929 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:15:04.151096 17282 external_mini_cluster.cc:1428] Setting key 12368a41d6c3d1e50178b848eb33a4fd
I20260809 16:15:04.151460 14480 heartbeater.cc:347] Connected to a master server at 127.16.224.190:46427
I20260809 16:15:04.151566 14480 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:04.151767 14480 heartbeater.cc:511] Master 127.16.224.190:46427 requested a full tablet report, sending...
I20260809 16:15:04.152128 13802 ts_manager.cc:194] Registered new tserver with Master: 0654a228dd064fa0b78f5ccd10b95722 (127.16.224.131:45389)
I20260809 16:15:04.152493 13802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:44839
I20260809 16:15:04.162714 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:15:04.168996 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.163969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.16.224.190:46427)
0809 16:15:04.164014 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.164016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.164118 (+ 102us) server_negotiation.cc:407] Connection header received
0809 16:15:04.164158 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.164159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.164168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.164210 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:04.164676 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.165138 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.165774 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.165937 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.165976 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.167095 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:04.167096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:04.167097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:04.167103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:04.168098 (+ 995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:04.168419 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:04.168420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:04.168420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:04.168464 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:04.168596 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:04.168597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:04.168597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:04.168705 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:04.168779 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.168861 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.168908 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:15:04.170361 13802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:15:04.171865 13802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:15:04.177857 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.174040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.16.224.129:45303)
0809 16:15:04.174240 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.174246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.174370 (+ 124us) server_negotiation.cc:407] Connection header received
0809 16:15:04.174409 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.174411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.174425 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.174482 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.174696 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.175821 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.176966 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.177475 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.177549 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.177608 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.177736 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.177778 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
I20260809 16:15:04.177845 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.173851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:45303 (local address 127.0.0.1:52812)
0809 16:15:04.174211 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.174222 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.174284 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.174590 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.174591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.174592 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.174625 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.174626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.175928 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.175929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.176899 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.176902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.177561 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.177562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.177639 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.177652 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.177715 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.177774 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":91,"threads_started":1}
I20260809 16:15:04.178014 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.174072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50936 (local address 127.16.224.131:45389)
0809 16:15:04.174250 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.174254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.174264 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:15:04.174303 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.174304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.174349 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.174418 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.174573 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.175472 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.176872 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.177516 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.177585 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.177634 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.177886 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.177924 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260809 16:15:04.178115 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.173844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:45389 (local address 127.0.0.1:50936)
0809 16:15:04.173999 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.174037 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.174099 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.174436 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.174439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.174441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.174509 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.174510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.175565 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.175566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.176804 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.176807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.177671 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.177672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.177747 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.177752 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.177797 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.177840 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":56,"threads_started":1}
I20260809 16:15:04.178220 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.173846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:35479 (local address 127.0.0.1:43204)
0809 16:15:04.174198 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.174207 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.174245 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.174573 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.174574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.174576 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.174626 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.174627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.175898 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.175899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.177004 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.177006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.177882 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.177882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.177947 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.177950 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.178013 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.178048 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":32,"thread_start_us":87,"threads_started":1}
I20260809 16:15:04.178273 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.174054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.16.224.130:35479)
0809 16:15:04.174344 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.174348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.174357 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:15:04.174408 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.174410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.174425 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.174498 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.174695 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.175795 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.177078 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.177747 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.177806 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.177851 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.178159 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.178202 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":74,"threads_started":1}
I20260809 16:15:04.178974 14374 tablet_service.cc:1467] Processing CreateTablet for tablet 0902b02773c8431283eacca1d9a83d8c (DEFAULT_TABLE table=test-table [id=18ce12ca11dc4b1892984d6824bf7b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:04.179122 14180 tablet_service.cc:1467] Processing CreateTablet for tablet 0902b02773c8431283eacca1d9a83d8c (DEFAULT_TABLE table=test-table [id=18ce12ca11dc4b1892984d6824bf7b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:04.179352 14374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0902b02773c8431283eacca1d9a83d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:04.179450 14180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0902b02773c8431283eacca1d9a83d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:04.181843 14511 tablet_bootstrap.cc:492] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Bootstrap starting.
I20260809 16:15:04.182276 13985 tablet_service.cc:1467] Processing CreateTablet for tablet 0902b02773c8431283eacca1d9a83d8c (DEFAULT_TABLE table=test-table [id=18ce12ca11dc4b1892984d6824bf7b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:04.182471 14515 tablet_bootstrap.cc:492] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Bootstrap starting.
I20260809 16:15:04.182513 14511 tablet_bootstrap.cc:654] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:04.182631 13985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0902b02773c8431283eacca1d9a83d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:04.183147 14511 log.cc:824] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:04.183192 14515 tablet_bootstrap.cc:654] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:04.183669 14515 log.cc:824] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:04.184032 14511 tablet_bootstrap.cc:492] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: No bootstrap required, opened a new log
I20260809 16:15:04.184096 14511 ts_tablet_manager.cc:1397] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:15:04.184412 14515 tablet_bootstrap.cc:492] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: No bootstrap required, opened a new log
I20260809 16:15:04.184486 14515 ts_tablet_manager.cc:1397] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:04.185514 14511 raft_consensus.cc:348] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.185753 14518 tablet_bootstrap.cc:492] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a: Bootstrap starting.
I20260809 16:15:04.185768 14511 raft_consensus.cc:374] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:04.185814 14511 raft_consensus.cc:729] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0654a228dd064fa0b78f5ccd10b95722, State: Initialized, Role: FOLLOWER
I20260809 16:15:04.186046 14511 consensus_queue.cc:260] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.186487 14518 tablet_bootstrap.cc:654] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:04.186487 14480 heartbeater.cc:503] Master 127.16.224.190:46427 was elected leader, sending a full tablet report...
I20260809 16:15:04.186491 14511 ts_tablet_manager.cc:1428] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:15:04.186383 14515 raft_consensus.cc:348] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.186569 14515 raft_consensus.cc:374] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:04.186654 14515 raft_consensus.cc:729] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb05c8e438f4feabd075c2c0f3ce02f, State: Initialized, Role: FOLLOWER
I20260809 16:15:04.186842 14515 consensus_queue.cc:260] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.187084 14518 log.cc:824] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:04.187127 14273 heartbeater.cc:503] Master 127.16.224.190:46427 was elected leader, sending a full tablet report...
I20260809 16:15:04.187433 14515 ts_tablet_manager.cc:1428] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:15:04.187857 14518 tablet_bootstrap.cc:492] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a: No bootstrap required, opened a new log
I20260809 16:15:04.187922 14518 ts_tablet_manager.cc:1397] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:15:04.189277 14518 raft_consensus.cc:348] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.189469 14518 raft_consensus.cc:374] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:04.189525 14518 raft_consensus.cc:729] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2554249f8dd245889ee61fc2962f168a, State: Initialized, Role: FOLLOWER
I20260809 16:15:04.189639 14518 consensus_queue.cc:260] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.189868 14518 ts_tablet_manager.cc:1428] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:15:04.189910 14084 heartbeater.cc:503] Master 127.16.224.190:46427 was elected leader, sending a full tablet report...
I20260809 16:15:04.216952 14524 raft_consensus.cc:482] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:15:04.217096 14524 raft_consensus.cc:504] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.217458 14524 leader_election.cc:288] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0654a228dd064fa0b78f5ccd10b95722 (127.16.224.131:45389), 5cb05c8e438f4feabd075c2c0f3ce02f (127.16.224.130:35479)
I20260809 16:15:04.220191 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.217688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:59593 (local address 127.16.224.130:35479)
0809 16:15:04.217749 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.217751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.217760 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:15:04.217789 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.217790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.217799 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.217861 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.218060 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.218602 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.219513 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.219880 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.219942 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.219951 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.220093 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.220126 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:15:04.220191 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.217753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:50321 (local address 127.16.224.131:45389)
0809 16:15:04.217795 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.217797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.217871 (+ 74us) server_negotiation.cc:407] Connection header received
0809 16:15:04.217918 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.217919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.217928 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.217986 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.218138 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.218630 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.219508 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.219879 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.219942 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.219951 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.220091 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.220127 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:15:04.220196 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.217607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:35479 (local address 127.16.224.129:59593)
0809 16:15:04.217662 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.217673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.217714 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.217875 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.217877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.217878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.217971 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.217974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.218678 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.218679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.219417 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.219420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.219952 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.219953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.220025 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.220028 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.220074 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.220116 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:15:04.220206 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.217607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:45389 (local address 127.16.224.129:50321)
0809 16:15:04.217807 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.217816 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.217870 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.218007 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.218007 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.218009 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.218078 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.218080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.218705 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.218706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.219421 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.219423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.219958 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.219959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.220022 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.220025 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.220071 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.220112 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":12,"thread_start_us":114,"threads_started":1}
I20260809 16:15:04.220541 14206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0902b02773c8431283eacca1d9a83d8c" candidate_uuid: "2554249f8dd245889ee61fc2962f168a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" is_pre_election: true
I20260809 16:15:04.220587 14404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0902b02773c8431283eacca1d9a83d8c" candidate_uuid: "2554249f8dd245889ee61fc2962f168a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0654a228dd064fa0b78f5ccd10b95722" is_pre_election: true
I20260809 16:15:04.220698 14206 raft_consensus.cc:2454] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2554249f8dd245889ee61fc2962f168a in term 0.
I20260809 16:15:04.220717 14404 raft_consensus.cc:2454] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2554249f8dd245889ee61fc2962f168a in term 0.
I20260809 16:15:04.220911 13922 leader_election.cc:302] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2554249f8dd245889ee61fc2962f168a, 5cb05c8e438f4feabd075c2c0f3ce02f; no voters:
I20260809 16:15:04.221036 14524 raft_consensus.cc:2781] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:15:04.221092 14524 raft_consensus.cc:482] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:15:04.221112 14524 raft_consensus.cc:3037] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:04.221798 14524 raft_consensus.cc:504] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.221930 14524 leader_election.cc:288] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [CANDIDATE]: Term 1 election: Requested vote from peers 0654a228dd064fa0b78f5ccd10b95722 (127.16.224.131:45389), 5cb05c8e438f4feabd075c2c0f3ce02f (127.16.224.130:35479)
I20260809 16:15:04.222091 14206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0902b02773c8431283eacca1d9a83d8c" candidate_uuid: "2554249f8dd245889ee61fc2962f168a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f"
I20260809 16:15:04.222134 14404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0902b02773c8431283eacca1d9a83d8c" candidate_uuid: "2554249f8dd245889ee61fc2962f168a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0654a228dd064fa0b78f5ccd10b95722"
I20260809 16:15:04.222167 14206 raft_consensus.cc:3037] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:04.222199 14404 raft_consensus.cc:3037] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:15:04.222379 14274 tablet.cc:2369] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:15:04.222965 14404 raft_consensus.cc:2454] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2554249f8dd245889ee61fc2962f168a in term 1.
I20260809 16:15:04.222985 14206 raft_consensus.cc:2454] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2554249f8dd245889ee61fc2962f168a in term 1.
I20260809 16:15:04.223157 13920 leader_election.cc:302] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0654a228dd064fa0b78f5ccd10b95722, 2554249f8dd245889ee61fc2962f168a; no voters:
I20260809 16:15:04.223244 14524 raft_consensus.cc:2781] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:04.223414 14524 raft_consensus.cc:686] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [term 1 LEADER]: Becoming Leader. State: Replica: 2554249f8dd245889ee61fc2962f168a, State: Running, Role: LEADER
I20260809 16:15:04.223510 14524 consensus_queue.cc:237] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } }
I20260809 16:15:04.224355 13801 catalog_manager.cc:5495] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2554249f8dd245889ee61fc2962f168a (127.16.224.129). New cstate: current_term: 1 leader_uuid: "2554249f8dd245889ee61fc2962f168a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2554249f8dd245889ee61fc2962f168a" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 45303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 } health_report { overall_health: UNKNOWN } } }
I20260809 16:15:04.239399 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.237296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.16.224.129:45303)
0809 16:15:04.237358 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.237360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.237368 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:04.237398 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.237399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.237415 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.237469 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:04.237577 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.238064 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.238803 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.238990 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.239036 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.239154 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:04.239235 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:04.239298 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.239318 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.239327 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:15:04.241080 14404 raft_consensus.cc:1261] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 1 FOLLOWER]: Refusing update from remote peer 2554249f8dd245889ee61fc2962f168a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:04.241084 14206 raft_consensus.cc:1261] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 1 FOLLOWER]: Refusing update from remote peer 2554249f8dd245889ee61fc2962f168a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:04.241422 14528 consensus_queue.cc:1035] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb05c8e438f4feabd075c2c0f3ce02f" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:04.241551 14524 consensus_queue.cc:1035] T 0902b02773c8431283eacca1d9a83d8c P 2554249f8dd245889ee61fc2962f168a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0654a228dd064fa0b78f5ccd10b95722" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 45389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:04.243672 14532 mvcc.cc:205] Tried to move back new op lower bound from 7316652458969108480 to 7316652458900049920. Current Snapshot: MvccSnapshot[applied={T|T < 7316652458969108480}]
I20260809 16:15:04.244055 14537 mvcc.cc:205] Tried to move back new op lower bound from 7316652458969108480 to 7316652458900049920. Current Snapshot: MvccSnapshot[applied={T|T < 7316652458969108480}]
I20260809 16:15:04.246364 13801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:04.246461 13801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:04.247198 13801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41baa2a9099c4a889d5229bd6ae434d9: Sending DeleteTablet for 3 replicas of tablet 0902b02773c8431283eacca1d9a83d8c
I20260809 16:15:04.247538 14374 tablet_service.cc:1514] Processing DeleteTablet for tablet 0902b02773c8431283eacca1d9a83d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:04 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:50936
I20260809 16:15:04.247537 14180 tablet_service.cc:1514] Processing DeleteTablet for tablet 0902b02773c8431283eacca1d9a83d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:04 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:43204
I20260809 16:15:04.247618 13985 tablet_service.cc:1514] Processing DeleteTablet for tablet 0902b02773c8431283eacca1d9a83d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:04 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:52812
I20260809 16:15:04.247785 14550 tablet_replica.cc:330] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: stopping tablet replica
I20260809 16:15:04.247836 14552 tablet_replica.cc:330] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: stopping tablet replica
I20260809 16:15:04.247932 14552 raft_consensus.cc:2229] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:04.248009 14550 raft_consensus.cc:2229] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:04.248056 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 13898
I20260809 16:15:04.248781 14550 raft_consensus.cc:2258] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:04.251174 14550 ts_tablet_manager.cc:1905] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:04.252803 14550 ts_tablet_manager.cc:1918] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:04.252864 14550 log.cc:1196] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/0902b02773c8431283eacca1d9a83d8c
I20260809 16:15:04.253108 14550 ts_tablet_manager.cc:1939] T 0902b02773c8431283eacca1d9a83d8c P 5cb05c8e438f4feabd075c2c0f3ce02f: Deleting consensus metadata
I20260809 16:15:04.253858 13784 catalog_manager.cc:4841] TS 5cb05c8e438f4feabd075c2c0f3ce02f (127.16.224.130:35479): tablet 0902b02773c8431283eacca1d9a83d8c (table test-table [id=18ce12ca11dc4b1892984d6824bf7b65]) successfully deleted
I20260809 16:15:04.258436 14552 raft_consensus.cc:2258] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:04.258850 14552 ts_tablet_manager.cc:1905] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:15:04.259743 13780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:45303 (error 108)
I20260809 16:15:04.260298 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.260163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:45303 (local address 127.0.0.1:52836)
0809 16:15:04.260219 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.260262 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:45303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:15:04.260315 14552 ts_tablet_manager.cc:1918] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:04.260334 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 14094
I20260809 16:15:04.260382 14552 log.cc:1196] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/0902b02773c8431283eacca1d9a83d8c
I20260809 16:15:04.264500 14552 ts_tablet_manager.cc:1939] T 0902b02773c8431283eacca1d9a83d8c P 0654a228dd064fa0b78f5ccd10b95722: Deleting consensus metadata
I20260809 16:15:04.265306 13781 catalog_manager.cc:4841] TS 0654a228dd064fa0b78f5ccd10b95722 (127.16.224.131:45389): tablet 0902b02773c8431283eacca1d9a83d8c (table test-table [id=18ce12ca11dc4b1892984d6824bf7b65]) successfully deleted
W20260809 16:15:04.265344 13780 catalog_manager.cc:4584] TS 2554249f8dd245889ee61fc2962f168a (127.16.224.129:45303): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0902b02773c8431283eacca1d9a83d8c: Network error: Client connection negotiation failed: client connection to 127.16.224.129:45303: connect: Connection refused (error 111)
I20260809 16:15:04.270684 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 14278
I20260809 16:15:04.283443 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 13753
2026-08-09T16:15:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2104 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T16:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:04Z Disabled control of system clock
I20260809 16:15:04.315627 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:36501
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:36501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:04.397820 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:04.398056 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:04.398078 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:04.399228 14561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:04.399271 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:04.399281 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:04.399293 14561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:04.399310 14561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:04.400606 14561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:36501
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:36501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:04.400801 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:04.401051 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:04.405611 14568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.405611 14571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.405622 14567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:04.405655 14561 server_base.cc:1034] running on GCE node
I20260809 16:15:04.406049 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:04.406361 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:04.407513 14561 hybrid_clock.cc:648] HybridClock initialized: now 1786292104407499 us; error 32 us; skew 500 ppm
I20260809 16:15:04.408703 14561 webserver.cc:460] Webserver started at http://127.16.224.190:45413/ using document root <none> and password file <none>
I20260809 16:15:04.408947 14561 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:04.409009 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:04.409139 14561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:04.410166 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "d6eb38ea031f09d786f8596136f290ef"
server_key_iv: "95d5bcc101aef152891ba839481749ed"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.410531 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "d6eb38ea031f09d786f8596136f290ef"
server_key_iv: "95d5bcc101aef152891ba839481749ed"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.411721 14561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:15:04.412367 14578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.412525 14561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:04.412611 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "d6eb38ea031f09d786f8596136f290ef"
server_key_iv: "95d5bcc101aef152891ba839481749ed"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.412676 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:04.431881 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:04.432163 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:04.432289 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:04.436252 14561 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:36501
I20260809 16:15:04.436318 14653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:36501 every 8 connection(s)
I20260809 16:15:04.436623 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:04.437602 14655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:04.440009 14655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b: Bootstrap starting.
I20260809 16:15:04.440016 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 14561
I20260809 16:15:04.440109 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:04.440295 17282 external_mini_cluster.cc:1428] Setting key fcc112c0293523fdacd2734b1cd8bac5
I20260809 16:15:04.440626 14655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:04.441156 14655 log.cc:824] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:04.441890 14655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b: No bootstrap required, opened a new log
I20260809 16:15:04.444002 14655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36501 } }
I20260809 16:15:04.444137 14655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:04.444159 14655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57a0e5cd3bf9405c8e7ecbed393c7c2b, State: Initialized, Role: FOLLOWER
I20260809 16:15:04.444276 14655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36501 } }
I20260809 16:15:04.444342 14655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:04.444363 14655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:04.444409 14655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:04.445017 14655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36501 } }
I20260809 16:15:04.445132 14655 leader_election.cc:302] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57a0e5cd3bf9405c8e7ecbed393c7c2b; no voters:
I20260809 16:15:04.445313 14655 leader_election.cc:288] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:04.445348 14663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:04.445449 14663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [term 1 LEADER]: Becoming Leader. State: Replica: 57a0e5cd3bf9405c8e7ecbed393c7c2b, State: Running, Role: LEADER
I20260809 16:15:04.445590 14655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:04.445613 14663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36501 } }
I20260809 16:15:04.445977 14663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57a0e5cd3bf9405c8e7ecbed393c7c2b. Latest consensus state: current_term: 1 leader_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36501 } } }
I20260809 16:15:04.446026 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.440761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56938 (local address 127.16.224.190:36501)
0809 16:15:04.440962 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.440969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.440998 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:15:04.441455 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.441465 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.441559 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.441685 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:04.441936 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.442884 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.443961 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.444431 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.444546 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.445210 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:04.445214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:04.445214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:04.445405 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:04.445449 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:04.445466 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:04.445569 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.445750 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.445834 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260809 16:15:04.446224 14663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:04.446149 14665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a0e5cd3bf9405c8e7ecbed393c7c2b" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 36501 } } }
I20260809 16:15:04.446346 14665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:04.446859 14680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:04.447461 14680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:04.448807 14680 catalog_manager.cc:1347] Generated new cluster ID: ca13061d12dc43cb8b1fba60e7b53c68
I20260809 16:15:04.448868 14680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:04.476562 14680 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:04.477092 14680 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:04.482797 14680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b: Generated new TSK 0
I20260809 16:15:04.482961 14680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:04.486675 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:36501
--builtin_ntp_servers=127.16.224.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:04.569734 14692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:04.569949 14692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:04.569972 14692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:04.571134 14692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:04.571180 14692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:04.571208 14692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:15:04.572592 14692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:04.572844 14692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:04.573160 14692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:04.578220 14699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.578241 14703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.578217 14701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:04.578373 14692 server_base.cc:1034] running on GCE node
I20260809 16:15:04.578704 14692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:04.578939 14692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:04.580087 14692 hybrid_clock.cc:648] HybridClock initialized: now 1786292104580068 us; error 30 us; skew 500 ppm
I20260809 16:15:04.581314 14692 webserver.cc:460] Webserver started at http://127.16.224.129:41797/ using document root <none> and password file <none>
I20260809 16:15:04.581554 14692 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:04.581609 14692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:04.581759 14692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:04.582917 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "9460a46680934d8993b8bbe61a8275b2"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "1d1cc01677183c8a9e5e0fea7a1d679b"
server_key_iv: "24643a0bbf7e7e8f35e35e3738625e96"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.583282 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "9460a46680934d8993b8bbe61a8275b2"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "1d1cc01677183c8a9e5e0fea7a1d679b"
server_key_iv: "24643a0bbf7e7e8f35e35e3738625e96"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.584560 14692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:04.585285 14711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.585467 14692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:04.585561 14692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "9460a46680934d8993b8bbe61a8275b2"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "1d1cc01677183c8a9e5e0fea7a1d679b"
server_key_iv: "24643a0bbf7e7e8f35e35e3738625e96"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.585623 14692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:04.604249 14692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:04.604538 14692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:04.604666 14692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:04.604885 14692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:04.605212 14692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:04.605257 14692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.605284 14692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:04.605300 14692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.610915 14692 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:35225
I20260809 16:15:04.610988 14869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:35225 every 8 connection(s)
I20260809 16:15:04.611263 14692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:15:04.612429 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 14692
I20260809 16:15:04.612535 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:15:04.612756 17282 external_mini_cluster.cc:1428] Setting key 3736ea3c5d3216a0b47425c050374db1
I20260809 16:15:04.613724 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:36501
--builtin_ntp_servers=127.16.224.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:15:04.616753 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.611927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:37307 (local address 127.16.224.190:36501)
0809 16:15:04.612003 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.612005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.612218 (+ 213us) server_negotiation.cc:407] Connection header received
0809 16:15:04.612826 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.612827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.612840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.612904 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:04.613716 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.614393 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.615412 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.615619 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.615675 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.616496 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:04.616497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:04.616497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:04.616502 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:04.616519 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:04.616522 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:04.616570 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.616674 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.616695 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:15:04.617002 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.611907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:36501 (local address 127.16.224.129:37307)
0809 16:15:04.612107 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.612141 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.612723 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.612978 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.612982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.612992 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:04.613628 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.613634 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.614485 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.614486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.615342 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.615345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.615689 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.615690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.615833 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.616337 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:04.616338 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:04.616375 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:04.616418 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:04.616592 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:04.616600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:04.616603 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.616672 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.616778 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260809 16:15:04.617501 14870 heartbeater.cc:347] Connected to a master server at 127.16.224.190:36501
I20260809 16:15:04.617589 14870 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:04.617910 14870 heartbeater.cc:511] Master 127.16.224.190:36501 requested a full tablet report, sending...
I20260809 16:15:04.618592 14600 ts_manager.cc:194] Registered new tserver with Master: 9460a46680934d8993b8bbe61a8275b2 (127.16.224.129:35225)
I20260809 16:15:04.619565 14600 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.129:37307
W20260809 16:15:04.696727 14873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:04.696929 14873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:04.696950 14873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:04.698028 14873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:04.698062 14873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:04.698084 14873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:15:04.699405 14873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:04.699592 14873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:04.699854 14873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:04.704787 14882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.704782 14883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.704782 14886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:04.705084 14873 server_base.cc:1034] running on GCE node
I20260809 16:15:04.705294 14873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:04.705523 14873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:04.706657 14873 hybrid_clock.cc:648] HybridClock initialized: now 1786292104706643 us; error 28 us; skew 500 ppm
I20260809 16:15:04.707921 14873 webserver.cc:460] Webserver started at http://127.16.224.130:34169/ using document root <none> and password file <none>
I20260809 16:15:04.708150 14873 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:04.708197 14873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:04.708334 14873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:04.709530 14873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "c55b957ef0984d7392117324021954f5"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "5aae75d18bb8b665862646d9b390af18"
server_key_iv: "0b0de46cc0c2aa63acd97070630b1d9d"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.709896 14873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "c55b957ef0984d7392117324021954f5"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "5aae75d18bb8b665862646d9b390af18"
server_key_iv: "0b0de46cc0c2aa63acd97070630b1d9d"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.711267 14873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:04.712000 14895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.712153 14873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:04.712239 14873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "c55b957ef0984d7392117324021954f5"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "5aae75d18bb8b665862646d9b390af18"
server_key_iv: "0b0de46cc0c2aa63acd97070630b1d9d"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.712313 14873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:04.725044 14873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:04.725345 14873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:04.725486 14873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:04.725706 14873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:04.726044 14873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:04.726079 14873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.726109 14873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:04.726132 14873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.731774 14873 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:36983
I20260809 16:15:04.731817 15057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:36983 every 8 connection(s)
I20260809 16:15:04.732129 14873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:15:04.736841 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.732817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:46213 (local address 127.16.224.190:36501)
0809 16:15:04.732902 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.732904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.733097 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:15:04.733570 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.733571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.733582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.733631 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:04.734260 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.734789 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.735718 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.735895 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.735938 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.736632 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:04.736634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:04.736634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:04.736638 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:04.736654 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:04.736658 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:04.736694 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.736756 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.736779 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 16:15:04.737043 15061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.732785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:36501 (local address 127.16.224.130:46213)
0809 16:15:04.732979 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.733014 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.733489 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.733686 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.733689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.733697 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:04.734197 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.734202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.734859 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.734859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.735656 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.735658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.735951 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.735952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.736065 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.736511 (+ 446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:04.736511 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:04.736538 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:04.736562 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:04.736702 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:04.736704 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:04.736707 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.736753 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.736832 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260809 16:15:04.737547 15058 heartbeater.cc:347] Connected to a master server at 127.16.224.190:36501
I20260809 16:15:04.737636 15058 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:04.737807 15058 heartbeater.cc:511] Master 127.16.224.190:36501 requested a full tablet report, sending...
I20260809 16:15:04.738183 14600 ts_manager.cc:194] Registered new tserver with Master: c55b957ef0984d7392117324021954f5 (127.16.224.130:36983)
I20260809 16:15:04.738566 14600 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.130:46213
I20260809 16:15:04.740000 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 14873
I20260809 16:15:04.740084 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:15:04.740249 17282 external_mini_cluster.cc:1428] Setting key 70845ffba1929c4fac0c6cf399ba8532
I20260809 16:15:04.741125 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:36501
--builtin_ntp_servers=127.16.224.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:04.823611 15063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:04.823802 15063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:04.823822 15063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:04.824884 15063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:04.824916 15063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:04.824939 15063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:15:04.826263 15063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:04.826488 15063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:04.826733 15063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:04.831491 15074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.831532 15068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:04.831491 15069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:04.831770 15063 server_base.cc:1034] running on GCE node
I20260809 16:15:04.831983 15063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:04.832212 15063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:04.833372 15063 hybrid_clock.cc:648] HybridClock initialized: now 1786292104833359 us; error 29 us; skew 500 ppm
I20260809 16:15:04.834640 15063 webserver.cc:460] Webserver started at http://127.16.224.131:39939/ using document root <none> and password file <none>
I20260809 16:15:04.834857 15063 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:04.834906 15063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:04.835034 15063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:04.836088 15063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "15f1f46d1ff64f79abf68fbe9ba03197"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "4c2c43a98dfdc2aefaf47f04981af59e"
server_key_iv: "61d23ef3f0b30d3abb71048db95b1005"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.836462 15063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "15f1f46d1ff64f79abf68fbe9ba03197"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "4c2c43a98dfdc2aefaf47f04981af59e"
server_key_iv: "61d23ef3f0b30d3abb71048db95b1005"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.837736 15063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:04.838501 15081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.838651 15063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:04.838752 15063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "15f1f46d1ff64f79abf68fbe9ba03197"
format_stamp: "Formatted at 2026-08-09 16:15:04 on dist-test-slave-fgb3"
server_key: "4c2c43a98dfdc2aefaf47f04981af59e"
server_key_iv: "61d23ef3f0b30d3abb71048db95b1005"
server_key_version: "encryptionkey@0"
I20260809 16:15:04.838820 15063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:04.853904 15063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:04.854219 15063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:04.854429 15063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:04.854662 15063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:04.855031 15063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:04.855070 15063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.855103 15063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:04.855119 15063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:04.860728 15063 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:34985
I20260809 16:15:04.860787 15251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:34985 every 8 connection(s)
I20260809 16:15:04.861106 15063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:15:04.866256 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.861821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:53353 (local address 127.16.224.190:36501)
0809 16:15:04.861895 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.861897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.862094 (+ 197us) server_negotiation.cc:407] Connection header received
0809 16:15:04.862683 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.862685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.862699 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.862752 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:04.863461 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.863959 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.864976 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.865171 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.865228 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.865998 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:04.865999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:04.866000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:04.866005 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:04.866022 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:04.866026 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:04.866069 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.866143 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.866173 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:15:04.866480 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.861769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:36501 (local address 127.16.224.131:53353)
0809 16:15:04.861977 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.862010 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.862578 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.862852 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.862856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.862864 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:04.863363 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.863368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.864053 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.864054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.864884 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.864887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.865248 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.865250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.865390 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.865860 (+ 470us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:04.865860 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:04.865889 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:04.865913 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:04.866075 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:04.866081 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:04.866084 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.866141 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.866246 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260809 16:15:04.866757 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 15063
I20260809 16:15:04.866849 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:15:04.866981 15252 heartbeater.cc:347] Connected to a master server at 127.16.224.190:36501
I20260809 16:15:04.867063 17282 external_mini_cluster.cc:1428] Setting key 66066983a7d7e884d0de552eb230dfb4
I20260809 16:15:04.867080 15252 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:04.867252 15252 heartbeater.cc:511] Master 127.16.224.190:36501 requested a full tablet report, sending...
I20260809 16:15:04.867733 14600 ts_manager.cc:194] Registered new tserver with Master: 15f1f46d1ff64f79abf68fbe9ba03197 (127.16.224.131:34985)
I20260809 16:15:04.868116 14600 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.131:53353
I20260809 16:15:04.878968 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:15:04.882892 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.880343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.16.224.190:36501)
0809 16:15:04.880414 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.880416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.880447 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:15:04.880502 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.880503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.880511 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.880554 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:04.880696 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.881176 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.881841 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.882022 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.882065 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.882665 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:04.882666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:04.882667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:04.882671 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:04.882685 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:04.882689 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:04.882763 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.882803 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.882813 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:15:04.884418 14600 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:15:04.885963 14600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:15:04.892499 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.888554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57574 (local address 127.16.224.129:35225)
0809 16:15:04.888736 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.888741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.888751 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:15:04.888791 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.888793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.888808 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.888886 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.889141 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.890234 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.891348 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.892083 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.892177 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.892281 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.892364 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.892406 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
I20260809 16:15:04.892514 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.888152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:36983 (local address 127.0.0.1:41676)
0809 16:15:04.888352 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.888390 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.888456 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.888794 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.888798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.888800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.888869 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.888871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.890208 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.890209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.891349 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.891352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.892175 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.892176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.892262 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.892279 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.892352 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.892418 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":23,"thread_start_us":90,"threads_started":1}
I20260809 16:15:04.892791 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.888390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.16.224.130:36983)
0809 16:15:04.888599 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.888603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.888615 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:15:04.888682 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.888685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.888703 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.888786 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.888965 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.890117 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.891427 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.892109 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.892181 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.892271 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.892660 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.892706 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260809 16:15:04.892514 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.888180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:35225 (local address 127.0.0.1:57574)
0809 16:15:04.888576 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.888585 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.888622 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.889008 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.889009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.889011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.889047 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.889048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.890342 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.890344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.891261 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.891264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.892172 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.892173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.892262 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.892279 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.892351 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.892418 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":33,"thread_start_us":119,"threads_started":1}
I20260809 16:15:04.893536 14961 tablet_service.cc:1467] Processing CreateTablet for tablet c13d8a26959c4326885c45225ddfb70b (DEFAULT_TABLE table=test-table [id=69aae91ea52f4c81b499fc541e332b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:04.893612 14774 tablet_service.cc:1467] Processing CreateTablet for tablet c13d8a26959c4326885c45225ddfb70b (DEFAULT_TABLE table=test-table [id=69aae91ea52f4c81b499fc541e332b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:04.893785 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.888153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:34985 (local address 127.0.0.1:39948)
0809 16:15:04.888472 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.888483 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.888539 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.889130 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.889131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.889133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.889175 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.889177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.890006 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.890007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.891349 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.891352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.893539 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.893540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.893598 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.893601 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.893656 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.893699 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":48,"thread_start_us":82,"threads_started":1}
I20260809 16:15:04.893810 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.888364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.16.224.131:34985)
0809 16:15:04.888924 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.888928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.888937 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:15:04.888973 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.888975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.888999 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.889064 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.889246 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.889923 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.892314 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.893337 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.893410 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.893501 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.893683 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.893727 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":82,"threads_started":1}
I20260809 16:15:04.893945 14774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c13d8a26959c4326885c45225ddfb70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:04.893946 14961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c13d8a26959c4326885c45225ddfb70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:04.894960 15157 tablet_service.cc:1467] Processing CreateTablet for tablet c13d8a26959c4326885c45225ddfb70b (DEFAULT_TABLE table=test-table [id=69aae91ea52f4c81b499fc541e332b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:04.895272 15157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c13d8a26959c4326885c45225ddfb70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:04.897504 15272 tablet_bootstrap.cc:492] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2: Bootstrap starting.
I20260809 16:15:04.897711 15271 tablet_bootstrap.cc:492] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Bootstrap starting.
I20260809 16:15:04.898371 15273 tablet_bootstrap.cc:492] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Bootstrap starting.
I20260809 16:15:04.898569 15271 tablet_bootstrap.cc:654] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:04.898870 15271 log.cc:824] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:04.899017 15272 tablet_bootstrap.cc:654] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:04.899039 15273 tablet_bootstrap.cc:654] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:04.899343 15272 log.cc:824] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:04.899343 15273 log.cc:824] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:04.899677 15271 tablet_bootstrap.cc:492] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: No bootstrap required, opened a new log
I20260809 16:15:04.899875 15271 ts_tablet_manager.cc:1397] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:15:04.900048 15273 tablet_bootstrap.cc:492] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: No bootstrap required, opened a new log
I20260809 16:15:04.900061 15272 tablet_bootstrap.cc:492] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2: No bootstrap required, opened a new log
I20260809 16:15:04.900121 15272 ts_tablet_manager.cc:1397] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:15:04.900121 15273 ts_tablet_manager.cc:1397] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:04.901849 15271 raft_consensus.cc:348] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.901976 15271 raft_consensus.cc:374] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:04.901995 15271 raft_consensus.cc:729] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55b957ef0984d7392117324021954f5, State: Initialized, Role: FOLLOWER
I20260809 16:15:04.902088 15271 consensus_queue.cc:260] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.902366 15271 ts_tablet_manager.cc:1428] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:15:04.902580 15273 raft_consensus.cc:348] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.902580 15272 raft_consensus.cc:348] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.902732 15273 raft_consensus.cc:374] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:04.902732 15272 raft_consensus.cc:374] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:04.902768 15272 raft_consensus.cc:729] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9460a46680934d8993b8bbe61a8275b2, State: Initialized, Role: FOLLOWER
I20260809 16:15:04.902766 15273 raft_consensus.cc:729] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f1f46d1ff64f79abf68fbe9ba03197, State: Initialized, Role: FOLLOWER
I20260809 16:15:04.902881 15272 consensus_queue.cc:260] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.902881 15273 consensus_queue.cc:260] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.903138 15273 ts_tablet_manager.cc:1428] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:15:04.903138 15272 ts_tablet_manager.cc:1428] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:15:04.903192 14870 heartbeater.cc:503] Master 127.16.224.190:36501 was elected leader, sending a full tablet report...
I20260809 16:15:04.903262 15058 heartbeater.cc:503] Master 127.16.224.190:36501 was elected leader, sending a full tablet report...
I20260809 16:15:04.903524 15252 heartbeater.cc:503] Master 127.16.224.190:36501 was elected leader, sending a full tablet report...
I20260809 16:15:04.967639 15279 raft_consensus.cc:482] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:15:04.967775 15279 raft_consensus.cc:504] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.968159 15279 leader_election.cc:288] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15f1f46d1ff64f79abf68fbe9ba03197 (127.16.224.131:34985), 9460a46680934d8993b8bbe61a8275b2 (127.16.224.129:35225)
I20260809 16:15:04.970991 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.968484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:60599 (local address 127.16.224.129:35225)
0809 16:15:04.968546 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.968548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.968559 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:15:04.968592 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.968594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.968608 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.968664 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.968837 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.969333 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.970303 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.970675 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.970732 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.970741 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.970883 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.970919 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:15:04.971002 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.968434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:40413 (local address 127.16.224.131:34985)
0809 16:15:04.968491 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.968494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.968618 (+ 124us) server_negotiation.cc:407] Connection header received
0809 16:15:04.968675 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.968676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.968686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.968743 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.968890 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.969373 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.970279 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.970691 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.970758 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.970766 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.970899 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.970941 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:15:04.970999 15061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.968361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:35225 (local address 127.16.224.130:60599)
0809 16:15:04.968417 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.968430 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.968479 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.968679 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.968680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.968682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.968757 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.968759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.969411 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.969411 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.970204 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.970207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.970744 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.970745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.970818 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.970821 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.970862 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.970907 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:15:04.971012 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.968374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:34985 (local address 127.16.224.130:40413)
0809 16:15:04.968543 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.968553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.968615 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.968750 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.968751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.968752 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.968825 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.968827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.969451 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.969452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.970194 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.970197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.970773 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.970774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.970831 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.970834 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.970883 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.970919 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260809 16:15:04.971242 15282 raft_consensus.cc:482] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:15:04.971351 15282 raft_consensus.cc:504] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.971383 15181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13d8a26959c4326885c45225ddfb70b" candidate_uuid: "c55b957ef0984d7392117324021954f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" is_pre_election: true
I20260809 16:15:04.971366 14801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13d8a26959c4326885c45225ddfb70b" candidate_uuid: "c55b957ef0984d7392117324021954f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9460a46680934d8993b8bbe61a8275b2" is_pre_election: true
I20260809 16:15:04.971521 14801 raft_consensus.cc:2454] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c55b957ef0984d7392117324021954f5 in term 0.
I20260809 16:15:04.971527 15181 raft_consensus.cc:2454] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c55b957ef0984d7392117324021954f5 in term 0.
I20260809 16:15:04.971761 14898 leader_election.cc:302] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9460a46680934d8993b8bbe61a8275b2, c55b957ef0984d7392117324021954f5; no voters:
I20260809 16:15:04.971881 15282 leader_election.cc:288] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15f1f46d1ff64f79abf68fbe9ba03197 (127.16.224.131:34985), c55b957ef0984d7392117324021954f5 (127.16.224.130:36983)
I20260809 16:15:04.971971 15279 raft_consensus.cc:2781] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:15:04.972028 15279 raft_consensus.cc:482] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:15:04.972052 15279 raft_consensus.cc:3037] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:04.972939 15279 raft_consensus.cc:504] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.973192 15279 leader_election.cc:288] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [CANDIDATE]: Term 1 election: Requested vote from peers 15f1f46d1ff64f79abf68fbe9ba03197 (127.16.224.131:34985), 9460a46680934d8993b8bbe61a8275b2 (127.16.224.129:35225)
I20260809 16:15:04.973192 15181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13d8a26959c4326885c45225ddfb70b" candidate_uuid: "c55b957ef0984d7392117324021954f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f1f46d1ff64f79abf68fbe9ba03197"
I20260809 16:15:04.973274 15181 raft_consensus.cc:3037] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:04.973462 14801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13d8a26959c4326885c45225ddfb70b" candidate_uuid: "c55b957ef0984d7392117324021954f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9460a46680934d8993b8bbe61a8275b2"
I20260809 16:15:04.973528 14801 raft_consensus.cc:3037] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:04.974189 15181 raft_consensus.cc:2454] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c55b957ef0984d7392117324021954f5 in term 1.
I20260809 16:15:04.974395 14801 raft_consensus.cc:2454] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c55b957ef0984d7392117324021954f5 in term 1.
I20260809 16:15:04.974400 14900 leader_election.cc:302] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f1f46d1ff64f79abf68fbe9ba03197, c55b957ef0984d7392117324021954f5; no voters:
I20260809 16:15:04.974570 15279 raft_consensus.cc:2781] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:04.974750 15279 raft_consensus.cc:686] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 1 LEADER]: Becoming Leader. State: Replica: c55b957ef0984d7392117324021954f5, State: Running, Role: LEADER
I20260809 16:15:04.974867 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.971843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:36983 (local address 127.16.224.129:38213)
0809 16:15:04.971886 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.971897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.971939 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.972090 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.972090 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.972092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.972145 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.972147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.972953 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.972954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.973965 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.973967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.974646 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.974647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.974714 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.974717 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.974760 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.974808 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:15:04.974893 15279 consensus_queue.cc:237] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } }
I20260809 16:15:04.975104 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.971892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:38213 (local address 127.16.224.130:36983)
0809 16:15:04.971924 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.971926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.971933 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:04.972005 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.972005 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.972014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.972055 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.972213 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.972867 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.974058 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.974554 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.974609 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.974617 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.975020 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.975055 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":17}
I20260809 16:15:04.975292 14985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13d8a26959c4326885c45225ddfb70b" candidate_uuid: "9460a46680934d8993b8bbe61a8275b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55b957ef0984d7392117324021954f5" is_pre_election: true
I20260809 16:15:04.975369 15286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.972050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:34985 (local address 127.16.224.129:56957)
0809 16:15:04.972236 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:04.972247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:04.972297 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:04.972430 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:04.972430 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:04.972432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:04.972490 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.972491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.973371 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.973372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.974431 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:04.974434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.975106 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.975107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:04.975179 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.975183 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:04.975258 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:15:04.975295 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260809 16:15:04.975376 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.972264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:56957 (local address 127.16.224.131:34985)
0809 16:15:04.972307 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.972309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.972316 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:04.972367 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.972367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.972375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.972426 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:04.972601 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.973228 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.974524 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.974995 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.975042 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.975050 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.975273 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.975323 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:04.975653 15181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13d8a26959c4326885c45225ddfb70b" candidate_uuid: "9460a46680934d8993b8bbe61a8275b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" is_pre_election: true
I20260809 16:15:04.975744 15181 raft_consensus.cc:2379] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9460a46680934d8993b8bbe61a8275b2 in current term 1: Already voted for candidate c55b957ef0984d7392117324021954f5 in this term.
I20260809 16:15:04.975972 14716 leader_election.cc:302] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9460a46680934d8993b8bbe61a8275b2; no voters: 15f1f46d1ff64f79abf68fbe9ba03197, c55b957ef0984d7392117324021954f5
I20260809 16:15:04.975911 14599 catalog_manager.cc:5495] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c55b957ef0984d7392117324021954f5 (127.16.224.130). New cstate: current_term: 1 leader_uuid: "c55b957ef0984d7392117324021954f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c55b957ef0984d7392117324021954f5" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 36983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 } health_report { overall_health: UNKNOWN } } }
I20260809 16:15:04.976178 15282 raft_consensus.cc:2726] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 16:15:04.982685 15059 tablet.cc:2369] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:15:04.983846 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:04.981399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41682 (local address 127.16.224.130:36983)
0809 16:15:04.981443 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:04.981445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:04.981452 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:04.981480 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:04.981480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:04.981497 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:04.981558 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:04.981679 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.982151 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.983046 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:04.983292 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:04.983354 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:04.983516 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:04.983613 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:04.983695 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:04.983745 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 16:15:04.983752 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:15:04.985436 15181 raft_consensus.cc:1261] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 1 FOLLOWER]: Refusing update from remote peer c55b957ef0984d7392117324021954f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:04.985455 14801 raft_consensus.cc:1261] T c13d8a26959c4326885c45225ddfb70b P 9460a46680934d8993b8bbe61a8275b2 [term 1 FOLLOWER]: Refusing update from remote peer c55b957ef0984d7392117324021954f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:04.985689 15279 consensus_queue.cc:1035] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f1f46d1ff64f79abf68fbe9ba03197" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 34985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:04.985827 15287 consensus_queue.cc:1035] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9460a46680934d8993b8bbe61a8275b2" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 35225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:04.988072 15293 mvcc.cc:205] Tried to move back new op lower bound from 7316652462018031616 to 7316652461978468352. Current Snapshot: MvccSnapshot[applied={T|T < 7316652462018031616}]
I20260809 16:15:04.988193 15296 mvcc.cc:205] Tried to move back new op lower bound from 7316652462018031616 to 7316652461978468352. Current Snapshot: MvccSnapshot[applied={T|T < 7316652462018031616}]
I20260809 16:15:04.990677 14600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:04.990758 14600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:04.991446 14600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57a0e5cd3bf9405c8e7ecbed393c7c2b: Sending DeleteTablet for 3 replicas of tablet c13d8a26959c4326885c45225ddfb70b
I20260809 16:15:04.991755 15157 tablet_service.cc:1514] Processing DeleteTablet for tablet c13d8a26959c4326885c45225ddfb70b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:04 UTC) from {username='slave'} at 127.0.0.1:39948
I20260809 16:15:04.991959 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 14692
I20260809 16:15:04.991971 14961 tablet_service.cc:1514] Processing DeleteTablet for tablet c13d8a26959c4326885c45225ddfb70b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:04 UTC) from {username='slave'} at 127.0.0.1:41676
I20260809 16:15:04.994521 15306 tablet_replica.cc:330] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: stopping tablet replica
I20260809 16:15:04.994663 15306 raft_consensus.cc:2229] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:15:04.994818 15306 raft_consensus.cc:2258] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:05.000345 15305 tablet_replica.cc:330] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: stopping tablet replica
I20260809 16:15:05.000484 15305 raft_consensus.cc:2229] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:05.000600 15305 raft_consensus.cc:2258] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:05.003050 15305 ts_tablet_manager.cc:1905] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:05.003296 15306 ts_tablet_manager.cc:1905] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:05.004735 15305 ts_tablet_manager.cc:1918] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:05.004824 15305 log.cc:1196] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/c13d8a26959c4326885c45225ddfb70b
I20260809 16:15:05.005064 15305 ts_tablet_manager.cc:1939] T c13d8a26959c4326885c45225ddfb70b P 15f1f46d1ff64f79abf68fbe9ba03197: Deleting consensus metadata
I20260809 16:15:05.005323 15306 ts_tablet_manager.cc:1918] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:05.005468 15306 log.cc:1196] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/c13d8a26959c4326885c45225ddfb70b
I20260809 16:15:05.005532 14582 catalog_manager.cc:4841] TS 15f1f46d1ff64f79abf68fbe9ba03197 (127.16.224.131:34985): tablet c13d8a26959c4326885c45225ddfb70b (table test-table [id=69aae91ea52f4c81b499fc541e332b08]) successfully deleted
I20260809 16:15:05.005867 15306 ts_tablet_manager.cc:1939] T c13d8a26959c4326885c45225ddfb70b P c55b957ef0984d7392117324021954f5: Deleting consensus metadata
I20260809 16:15:05.006345 14579 catalog_manager.cc:4841] TS c55b957ef0984d7392117324021954f5 (127.16.224.130:36983): tablet c13d8a26959c4326885c45225ddfb70b (table test-table [id=69aae91ea52f4c81b499fc541e332b08]) successfully deleted
W20260809 16:15:05.008080 14581 connection.cc:535] client connection to 127.16.224.129:35225 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:15:05.008188 14581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:15:05.008203 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 14873
I20260809 16:15:05.012763 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:05.012612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:35225 (local address 127.0.0.1:57576)
0809 16:15:05.012675 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:05.012721 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 16:15:05.012883 14581 catalog_manager.cc:4584] TS 9460a46680934d8993b8bbe61a8275b2 (127.16.224.129:35225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c13d8a26959c4326885c45225ddfb70b: Network error: Client connection negotiation failed: client connection to 127.16.224.129:35225: connect: Connection refused (error 111)
I20260809 16:15:05.022087 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 15063
I20260809 16:15:05.039433 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 14561
2026-08-09T16:15:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (757 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:05 dist-test-slave-fgb3 krb5kdc[15312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:05 dist-test-slave-fgb3 krb5kdc[15312](info): set up 2 sockets
Aug 09 16:15:05 dist-test-slave-fgb3 krb5kdc[15312](info): commencing operation
krb5kdc: starting...
W20260809 16:15:06.183149 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:06.297904 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44525
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:44525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:06.383108 15335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:06.383312 15335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:06.383342 15335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:06.384444 15335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:06.384486 15335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:06.384505 15335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:06.384519 15335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:06.384536 15335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:06.385983 15335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:44525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:44525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:06.386215 15335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:06.386456 15335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:06.391085 15346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:06.391206 15335 server_base.cc:1034] running on GCE node
W20260809 16:15:06.391090 15343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:06.391258 15344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:06.391600 15335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:06.391894 15335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:06.393033 15335 hybrid_clock.cc:648] HybridClock initialized: now 1786292106393019 us; error 26 us; skew 500 ppm
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:06.395416 15335 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:06.395824 15335 webserver.cc:460] Webserver started at http://127.16.224.190:37561/ using document root <none> and password file <none>
I20260809 16:15:06.396029 15335 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:06.396077 15335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:06.396202 15335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:06.397269 15335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "a372d110243f4f279caf3722145acb6e"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "f5995fb2671f9f7a1b74ea5ce075de4b"
server_key_iv: "53f06f54e6d3118b409681d1d813e51c"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.397617 15335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "a372d110243f4f279caf3722145acb6e"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "f5995fb2671f9f7a1b74ea5ce075de4b"
server_key_iv: "53f06f54e6d3118b409681d1d813e51c"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.399044 15335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:06.399765 15357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.399948 15335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:06.400039 15335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "a372d110243f4f279caf3722145acb6e"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "f5995fb2671f9f7a1b74ea5ce075de4b"
server_key_iv: "53f06f54e6d3118b409681d1d813e51c"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.400101 15335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:06.407874 15335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:06.409901 15335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:06.410041 15335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:06.414000 15335 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:44525
I20260809 16:15:06.414060 15437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:44525 every 8 connection(s)
I20260809 16:15:06.414402 15335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:06.415284 15440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:06.417647 15440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e: Bootstrap starting.
I20260809 16:15:06.418229 15440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:06.418533 15440 log.cc:824] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:06.419206 15440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e: No bootstrap required, opened a new log
I20260809 16:15:06.420785 15440 raft_consensus.cc:348] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372d110243f4f279caf3722145acb6e" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44525 } }
I20260809 16:15:06.420934 15440 raft_consensus.cc:374] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:06.420956 15440 raft_consensus.cc:729] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a372d110243f4f279caf3722145acb6e, State: Initialized, Role: FOLLOWER
I20260809 16:15:06.421080 15440 consensus_queue.cc:260] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372d110243f4f279caf3722145acb6e" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44525 } }
I20260809 16:15:06.421149 15440 raft_consensus.cc:388] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:06.421170 15440 raft_consensus.cc:482] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:06.421197 15440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:06.421693 15440 raft_consensus.cc:504] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372d110243f4f279caf3722145acb6e" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44525 } }
I20260809 16:15:06.421785 15440 leader_election.cc:302] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a372d110243f4f279caf3722145acb6e; no voters:
I20260809 16:15:06.421936 15440 leader_election.cc:288] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:06.421983 15443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:06.422164 15443 raft_consensus.cc:686] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [term 1 LEADER]: Becoming Leader. State: Replica: a372d110243f4f279caf3722145acb6e, State: Running, Role: LEADER
I20260809 16:15:06.422192 15440 sys_catalog.cc:564] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:06.422420 15443 consensus_queue.cc:237] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372d110243f4f279caf3722145acb6e" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44525 } }
I20260809 16:15:06.422497 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 15335
I20260809 16:15:06.422613 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:06.422808 17282 external_mini_cluster.cc:1428] Setting key dfb375984d35b550315ec076ca5ff461
I20260809 16:15:06.422858 15445 sys_catalog.cc:455] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a372d110243f4f279caf3722145acb6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372d110243f4f279caf3722145acb6e" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44525 } } }
I20260809 16:15:06.422931 15445 sys_catalog.cc:458] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:06.423065 15446 sys_catalog.cc:455] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a372d110243f4f279caf3722145acb6e. Latest consensus state: current_term: 1 leader_uuid: "a372d110243f4f279caf3722145acb6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a372d110243f4f279caf3722145acb6e" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 44525 } } }
I20260809 16:15:06.423168 15453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:06.423209 15446 sys_catalog.cc:458] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:06.423770 15453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:06.425577 15453 catalog_manager.cc:1347] Generated new cluster ID: 68f4c325746142afaa7b4156d77f31a6
I20260809 16:15:06.425638 15453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:06.435580 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.423492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.16.224.190:44525)
0809 16:15:06.424169 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.424178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.424212 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:15:06.424487 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.424497 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.424561 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.424697 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:06.425544 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.427002 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.428120 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.428701 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.428803 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.431734 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:06.431739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:06.431739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:06.431752 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:06.433790 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.434213 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.434216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.434216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.434362 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.434551 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.434552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.434552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.434905 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:06.435062 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.435163 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.435314 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":68,"threads_started":1}
I20260809 16:15:06.445026 15453 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:06.445609 15453 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:06.451763 15453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e: Generated new TSK 0
I20260809 16:15:06.451963 15453 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.224.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
Entry for principal kudu/127.16.224.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab.
I20260809 16:15:06.483206 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44525
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:06.564961 15472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:06.565160 15472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:06.565191 15472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:06.566358 15472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:06.566399 15472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:06.566434 15472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:15:06.567873 15472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.129.keytab
--principal=kudu/127.16.224.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:06.568120 15472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:06.568322 15472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:06.572983 15482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:06.573046 15480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:06.573004 15484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:06.573316 15472 server_base.cc:1034] running on GCE node
I20260809 16:15:06.573489 15472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:06.573669 15472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:06.574806 15472 hybrid_clock.cc:648] HybridClock initialized: now 1786292106574791 us; error 27 us; skew 500 ppm
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:06.577160 15472 init.cc:377] Logged in from keytab as kudu/127.16.224.129@KRBTEST.COM (short username kudu)
I20260809 16:15:06.577569 15472 webserver.cc:460] Webserver started at http://127.16.224.129:40013/ using document root <none> and password file <none>
I20260809 16:15:06.577787 15472 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:06.577852 15472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:06.577983 15472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:06.579046 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "65fabf990b834552aad823f7bc88a258"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "d83d4ed68aae3918a41fe84856040870"
server_key_iv: "df7f992d0c7b3c1d2581cea89fe3e6f8"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.579370 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "65fabf990b834552aad823f7bc88a258"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "d83d4ed68aae3918a41fe84856040870"
server_key_iv: "df7f992d0c7b3c1d2581cea89fe3e6f8"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.580601 15472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:06.581234 15492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.581384 15472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:06.581470 15472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "65fabf990b834552aad823f7bc88a258"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "d83d4ed68aae3918a41fe84856040870"
server_key_iv: "df7f992d0c7b3c1d2581cea89fe3e6f8"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.581530 15472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:06.597194 15472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:06.599368 15472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:06.599512 15472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:06.599730 15472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:06.600040 15472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:06.600076 15472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.600106 15472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:06.600122 15472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.605819 15472 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:44279
I20260809 16:15:06.605881 15666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:44279 every 8 connection(s)
I20260809 16:15:06.606172 15472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:15:06.608341 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 15472
I20260809 16:15:06.608446 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:15:06.608639 17282 external_mini_cluster.cc:1428] Setting key f21764fca08413328e35c2627c2e225a
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.129@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:06.615461 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.606865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:45765 (local address 127.16.224.190:44525)
0809 16:15:06.606926 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.606928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.607124 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:15:06.607646 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.607647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.607665 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.607719 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:06.608691 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.609408 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.610413 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.610685 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.610740 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.612853 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:06.612855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:06.612855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:06.612862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:06.614145 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.614559 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.614560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.614560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.614620 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.614841 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.614842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.614842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.614956 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:06.615013 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.615277 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.615369 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:15:06.615600 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.606837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44525 (local address 127.16.224.129:45765)
0809 16:15:06.607007 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:06.607045 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:06.607543 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:06.607815 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:06.607819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:06.608364 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:06.608598 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.608605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.609492 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.609493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.610344 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.610347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.610761 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.610771 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.610915 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.611392 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:06.611400 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:06.612756 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:06.614224 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:06.614231 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:06.614234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:06.614492 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:06.614673 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:06.614673 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:06.614674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:06.614786 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:06.615011 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:06.615018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:06.615150 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:15:06.615237 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:15:06.615388 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260809 16:15:06.616092 15669 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44525
I20260809 16:15:06.616184 15669 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:06.616356 15669 heartbeater.cc:511] Master 127.16.224.190:44525 requested a full tablet report, sending...
I20260809 16:15:06.616938 15380 ts_manager.cc:194] Registered new tserver with Master: 65fabf990b834552aad823f7bc88a258 (127.16.224.129:44279)
WARNING: no policy specified for kudu/127.16.224.130@KRBTEST.COM; defaulting to no policy
I20260809 16:15:06.617586 15380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.129@KRBTEST.COM'} at 127.16.224.129:45765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
Entry for principal kudu/127.16.224.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab.
I20260809 16:15:06.633724 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44525
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:06.714035 15675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:06.714254 15675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:06.714286 15675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:06.715440 15675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:06.715485 15675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:06.715519 15675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:15:06.716991 15675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.130.keytab
--principal=kudu/127.16.224.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:06.717238 15675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:06.717458 15675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:06.722239 15685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:06.722429 15675 server_base.cc:1034] running on GCE node
W20260809 16:15:06.722229 15684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:06.722230 15687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:06.722675 15675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:06.722931 15675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:06.724085 15675 hybrid_clock.cc:648] HybridClock initialized: now 1786292106724070 us; error 32 us; skew 500 ppm
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:06.726538 15675 init.cc:377] Logged in from keytab as kudu/127.16.224.130@KRBTEST.COM (short username kudu)
I20260809 16:15:06.726974 15675 webserver.cc:460] Webserver started at http://127.16.224.130:43763/ using document root <none> and password file <none>
I20260809 16:15:06.727169 15675 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:06.727216 15675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:06.727344 15675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:06.728374 15675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "87c568b1793c4e86adf57c8c87cde0be"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "25ac895fc1786fe3d5a7719f41ac087c"
server_key_iv: "ba90e9fd9220581b2352227c5e191e4d"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.728708 15675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "87c568b1793c4e86adf57c8c87cde0be"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "25ac895fc1786fe3d5a7719f41ac087c"
server_key_iv: "ba90e9fd9220581b2352227c5e191e4d"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.729905 15675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:06.730598 15700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.730747 15675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:06.730835 15675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "87c568b1793c4e86adf57c8c87cde0be"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "25ac895fc1786fe3d5a7719f41ac087c"
server_key_iv: "ba90e9fd9220581b2352227c5e191e4d"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.730894 15675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:06.747995 15675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:06.750139 15675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:06.750290 15675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:06.750551 15675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:06.750876 15675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:06.750911 15675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.750942 15675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:06.750960 15675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.756516 15675 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:33517
I20260809 16:15:06.756577 15865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:33517 every 8 connection(s)
I20260809 16:15:06.756870 15675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:15:06.758338 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 15675
I20260809 16:15:06.758456 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:15:06.758635 17282 external_mini_cluster.cc:1428] Setting key 0f86a375eb5245c9ff8d5bb56b862256
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.130@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:06.766309 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.757549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:49951 (local address 127.16.224.190:44525)
0809 16:15:06.757594 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.757596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.757792 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:15:06.758365 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.758367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.758381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.758431 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:06.759561 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.760039 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.761042 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.761238 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.761292 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.763488 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:06.763490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:06.763490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:06.763498 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:06.765006 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.765389 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.765391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.765391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.765455 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.765701 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.765702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.765702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.765838 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:06.765909 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.766120 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.766210 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:06.766399 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.757500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44525 (local address 127.16.224.130:49951)
0809 16:15:06.757681 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:06.757716 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:06.758220 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:06.758523 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:06.758528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:06.759227 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:06.759479 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.759485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.760122 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.760124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.760963 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.760965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.761305 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.761306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.761442 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.761894 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:06.761903 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:06.763388 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:06.765094 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:06.765101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:06.765103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:06.765309 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:06.765530 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:06.765531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:06.765531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:06.765643 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:06.765897 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:06.765899 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:06.766008 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 16:15:06.766082 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:15:06.766199 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260809 16:15:06.766881 15868 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44525
I20260809 16:15:06.766973 15868 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:06.767139 15868 heartbeater.cc:511] Master 127.16.224.190:44525 requested a full tablet report, sending...
I20260809 16:15:06.767536 15380 ts_manager.cc:194] Registered new tserver with Master: 87c568b1793c4e86adf57c8c87cde0be (127.16.224.130:33517)
I20260809 16:15:06.767899 15380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.130@KRBTEST.COM'} at 127.16.224.130:49951
WARNING: no policy specified for kudu/127.16.224.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
Entry for principal kudu/127.16.224.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab.
I20260809 16:15:06.785072 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44525
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:06.865907 15877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:06.866111 15877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:06.866140 15877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:06.867308 15877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:06.867353 15877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:06.867388 15877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:15:06.868832 15877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.131.keytab
--principal=kudu/127.16.224.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:06.869081 15877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:06.869297 15877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:06.873997 15887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:06.874217 15884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:06.874161 15889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:06.874555 15877 server_base.cc:1034] running on GCE node
I20260809 16:15:06.874761 15877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:06.874972 15877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:06.876134 15877 hybrid_clock.cc:648] HybridClock initialized: now 1786292106876113 us; error 34 us; skew 500 ppm
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:06.878543 15877 init.cc:377] Logged in from keytab as kudu/127.16.224.131@KRBTEST.COM (short username kudu)
I20260809 16:15:06.878954 15877 webserver.cc:460] Webserver started at http://127.16.224.131:39151/ using document root <none> and password file <none>
I20260809 16:15:06.879170 15877 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:06.879217 15877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:06.879343 15877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:06.880370 15877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "670bb48be5874ae19564bb71949401bc"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "d03258ff95814395957da674cbe53992"
server_key_iv: "bec6deb37d0f681ebb68cb3500a0beba"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.880712 15877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "670bb48be5874ae19564bb71949401bc"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "d03258ff95814395957da674cbe53992"
server_key_iv: "bec6deb37d0f681ebb68cb3500a0beba"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.881955 15877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:06.882658 15900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.882802 15877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.882887 15877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "670bb48be5874ae19564bb71949401bc"
format_stamp: "Formatted at 2026-08-09 16:15:06 on dist-test-slave-fgb3"
server_key: "d03258ff95814395957da674cbe53992"
server_key_iv: "bec6deb37d0f681ebb68cb3500a0beba"
server_key_version: "encryptionkey@0"
I20260809 16:15:06.882948 15877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:06.902695 15877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:06.904769 15877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:06.904919 15877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:06.905133 15877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:06.905438 15877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:06.905473 15877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.905503 15877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:06.905529 15877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:06.911036 15877 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:36889
I20260809 16:15:06.911110 16068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:36889 every 8 connection(s)
I20260809 16:15:06.911410 15877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:15:06 dist-test-slave-fgb3 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292106, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.131@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:06.920094 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 15877
I20260809 16:15:06.920154 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.912114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:40205 (local address 127.16.224.190:44525)
0809 16:15:06.912173 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.912176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.912374 (+ 198us) server_negotiation.cc:407] Connection header received
0809 16:15:06.912838 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.912839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.912848 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.912911 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:06.913801 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.914303 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.915404 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.915596 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.915646 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.917546 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:06.917548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:06.917548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:06.917555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:06.918835 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.919268 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.919269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.919270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.919322 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.919549 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.919550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.919550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.919662 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:06.919719 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.919984 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.920064 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:15:06.920203 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:15:06.920312 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.912089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:44525 (local address 127.16.224.131:40205)
0809 16:15:06.912242 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:06.912285 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:06.912761 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:06.912971 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:06.912975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:06.913528 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:06.913728 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.913733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.914485 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.914486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.915329 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.915332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.915655 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.915656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.915750 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.916148 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:06.916156 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:06.917466 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:06.918919 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:06.918926 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:06.918928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:06.919198 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:06.919390 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:06.919390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:06.919391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:06.919491 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:06.919718 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:06.919725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:06.919857 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:15:06.919949 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:15:06.920102 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260809 16:15:06.920388 17282 external_mini_cluster.cc:1428] Setting key fa1872d5bfab69bfbf578c5ee1cf13b8
I20260809 16:15:06.920871 16071 heartbeater.cc:347] Connected to a master server at 127.16.224.190:44525
I20260809 16:15:06.920974 16071 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:06.921157 16071 heartbeater.cc:511] Master 127.16.224.190:44525 requested a full tablet report, sending...
I20260809 16:15:06.921562 15379 ts_manager.cc:194] Registered new tserver with Master: 670bb48be5874ae19564bb71949401bc (127.16.224.131:36889)
I20260809 16:15:06.921970 15379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.224.131@KRBTEST.COM'} at 127.16.224.131:40205
I20260809 16:15:06.932056 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:15:06.938829 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.933416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56716 (local address 127.16.224.190:44525)
0809 16:15:06.933472 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.933474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.933530 (+ 56us) server_negotiation.cc:407] Connection header received
0809 16:15:06.933603 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.933604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.933613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.933658 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:06.934198 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.934771 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.935471 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.935662 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.935715 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.936765 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:06.936767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:06.936767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:06.936774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:06.937739 (+ 965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.938050 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.938051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.938052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.938104 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:06.938258 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:06.938258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:06.938259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:06.938451 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:06.938540 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.938656 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.938732 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:15:06.940320 15379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:15:06.941864 15379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:15:06.947919 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.944277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49338 (local address 127.16.224.130:33517)
0809 16:15:06.944488 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.944492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.944503 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:15:06.944546 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.944549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.944564 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.944647 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:06.944907 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.945689 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.946902 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.947488 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.947568 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.947628 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.947793 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.947835 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":106,"threads_started":1}
I20260809 16:15:06.947908 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.944032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:33517 (local address 127.0.0.1:49338)
0809 16:15:06.944213 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:06.944256 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:06.944327 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:06.944741 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:06.944746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:06.944749 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:06.944819 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.944821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.945786 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.945787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.946803 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.946806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.947578 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.947579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.947681 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.947697 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 16:15:06.947768 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:15:06.947829 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":64,"threads_started":1}
I20260809 16:15:06.948498 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.944445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45408 (local address 127.16.224.131:36889)
0809 16:15:06.944652 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.944657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.944667 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:15:06.944710 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.944712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.944728 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.944886 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:06.945118 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.946266 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.947345 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.948068 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.948139 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.948192 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.948377 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.948427 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
I20260809 16:15:06.948812 15770 tablet_service.cc:1467] Processing CreateTablet for tablet 930b8b326ba849039ad8177307e069d2 (DEFAULT_TABLE table=test-table [id=74900b229c03488eb616f76ab29822a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:06.949168 15770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930b8b326ba849039ad8177307e069d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:06.949606 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.944034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44279 (local address 127.0.0.1:46458)
0809 16:15:06.944581 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:06.944590 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:06.944683 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:06.944981 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:06.944982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:06.944985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:06.945034 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.945036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.947350 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.947351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.948506 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.948509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.949420 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.949421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.949472 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.949476 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:06.949522 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:15:06.949555 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260809 16:15:06.949708 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.944229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.16.224.129:44279)
0809 16:15:06.944391 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:06.944396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:06.944653 (+ 257us) server_negotiation.cc:407] Connection header received
0809 16:15:06.944792 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:06.944795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:06.944812 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:06.944885 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:06.945135 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.946278 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.948651 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.949241 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.949325 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.949383 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:06.949579 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 16:15:06.949628 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260809 16:15:06.950479 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:06.944034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:36889 (local address 127.0.0.1:45408)
0809 16:15:06.944469 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:06.944478 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:06.944519 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:06.944975 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:06.944977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:06.944979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:06.945032 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.945034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.946402 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.946404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.947265 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:06.947268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:06.948230 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:06.948232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:06.948291 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:06.948295 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:06.948344 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:15:06.950428 (+ 2084us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260809 16:15:06.950454 15570 tablet_service.cc:1467] Processing CreateTablet for tablet 930b8b326ba849039ad8177307e069d2 (DEFAULT_TABLE table=test-table [id=74900b229c03488eb616f76ab29822a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:06.950790 15570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930b8b326ba849039ad8177307e069d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:06.951128 15970 tablet_service.cc:1467] Processing CreateTablet for tablet 930b8b326ba849039ad8177307e069d2 (DEFAULT_TABLE table=test-table [id=74900b229c03488eb616f76ab29822a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:06.951465 15970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930b8b326ba849039ad8177307e069d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:06.952566 16093 tablet_bootstrap.cc:492] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Bootstrap starting.
I20260809 16:15:06.953334 16093 tablet_bootstrap.cc:654] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:06.953786 16094 tablet_bootstrap.cc:492] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: Bootstrap starting.
I20260809 16:15:06.954005 16093 log.cc:824] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:06.954689 16096 tablet_bootstrap.cc:492] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Bootstrap starting.
I20260809 16:15:06.954939 16093 tablet_bootstrap.cc:492] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: No bootstrap required, opened a new log
I20260809 16:15:06.955003 16093 ts_tablet_manager.cc:1397] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:15:06.955446 16096 tablet_bootstrap.cc:654] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:06.955614 16094 tablet_bootstrap.cc:654] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:06.956014 16096 log.cc:824] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:06.956208 16094 log.cc:824] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:06.956707 16096 tablet_bootstrap.cc:492] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: No bootstrap required, opened a new log
I20260809 16:15:06.956775 16096 ts_tablet_manager.cc:1397] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:15:06.956827 16093 raft_consensus.cc:348] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:06.956982 16093 raft_consensus.cc:374] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:06.957024 16093 raft_consensus.cc:729] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c568b1793c4e86adf57c8c87cde0be, State: Initialized, Role: FOLLOWER
I20260809 16:15:06.957140 16093 consensus_queue.cc:260] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:06.957402 16093 ts_tablet_manager.cc:1428] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:15:06.957427 15868 heartbeater.cc:503] Master 127.16.224.190:44525 was elected leader, sending a full tablet report...
I20260809 16:15:06.958634 16096 raft_consensus.cc:348] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:06.958835 16094 tablet_bootstrap.cc:492] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: No bootstrap required, opened a new log
I20260809 16:15:06.958837 16096 raft_consensus.cc:374] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:06.958882 16096 raft_consensus.cc:729] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670bb48be5874ae19564bb71949401bc, State: Initialized, Role: FOLLOWER
I20260809 16:15:06.958899 16094 ts_tablet_manager.cc:1397] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 16:15:06.958993 16096 consensus_queue.cc:260] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:06.960487 16094 raft_consensus.cc:348] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:06.960601 16096 ts_tablet_manager.cc:1428] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 16:15:06.960628 16071 heartbeater.cc:503] Master 127.16.224.190:44525 was elected leader, sending a full tablet report...
I20260809 16:15:06.960659 16094 raft_consensus.cc:374] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:06.960691 16094 raft_consensus.cc:729] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65fabf990b834552aad823f7bc88a258, State: Initialized, Role: FOLLOWER
I20260809 16:15:06.960800 16094 consensus_queue.cc:260] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:06.961067 16094 ts_tablet_manager.cc:1428] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:15:06.961118 15669 heartbeater.cc:503] Master 127.16.224.190:44525 was elected leader, sending a full tablet report...
W20260809 16:15:07.007439 15872 tablet.cc:2369] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:15:07.092727 16103 raft_consensus.cc:482] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:15:07.092861 16103 raft_consensus.cc:504] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:07.093185 16103 leader_election.cc:288] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 670bb48be5874ae19564bb71949401bc (127.16.224.131:36889), 65fabf990b834552aad823f7bc88a258 (127.16.224.129:44279)
I20260809 16:15:07.095965 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.093442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:54715 (local address 127.16.224.131:36889)
0809 16:15:07.093486 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.093488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.093497 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:15:07.093526 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.093527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.093536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.093601 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:07.093896 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.094381 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.095260 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.095658 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.095720 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.095730 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.095863 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.095900 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:15:07.095965 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.093358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:36889 (local address 127.16.224.130:54715)
0809 16:15:07.093407 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.093418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.093470 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.093621 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.093623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.093625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:07.093693 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.093695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.094458 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.094459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.095181 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.095182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.095733 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.095734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.095805 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.095808 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.095852 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.095890 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:15:07.096076 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.093438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:34365 (local address 127.16.224.129:44279)
0809 16:15:07.093504 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.093507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.093768 (+ 261us) server_negotiation.cc:407] Connection header received
0809 16:15:07.093803 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.093804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.093814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.093853 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:07.094002 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.094528 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.095402 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.095773 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.095836 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.095845 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.095978 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.096016 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:15:07.096083 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.093358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44279 (local address 127.16.224.130:34365)
0809 16:15:07.093680 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.093688 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.093736 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.093867 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.093867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.093868 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:07.093929 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.093931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.094599 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.094600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.095319 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.095321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.095849 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.095850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.095909 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.095913 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.095963 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.095999 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":158,"thread_start_us":101,"threads_started":1}
I20260809 16:15:07.096333 16000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930b8b326ba849039ad8177307e069d2" candidate_uuid: "87c568b1793c4e86adf57c8c87cde0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670bb48be5874ae19564bb71949401bc" is_pre_election: true
I20260809 16:15:07.096499 16000 raft_consensus.cc:2454] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87c568b1793c4e86adf57c8c87cde0be in term 0.
I20260809 16:15:07.096483 15600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930b8b326ba849039ad8177307e069d2" candidate_uuid: "87c568b1793c4e86adf57c8c87cde0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fabf990b834552aad823f7bc88a258" is_pre_election: true
I20260809 16:15:07.096657 15600 raft_consensus.cc:2454] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87c568b1793c4e86adf57c8c87cde0be in term 0.
I20260809 16:15:07.096725 15703 leader_election.cc:302] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 670bb48be5874ae19564bb71949401bc, 87c568b1793c4e86adf57c8c87cde0be; no voters:
I20260809 16:15:07.096853 16103 raft_consensus.cc:2781] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:15:07.096909 16103 raft_consensus.cc:482] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:15:07.096932 16103 raft_consensus.cc:3037] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:07.097613 16103 raft_consensus.cc:504] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:07.097726 16103 leader_election.cc:288] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [CANDIDATE]: Term 1 election: Requested vote from peers 670bb48be5874ae19564bb71949401bc (127.16.224.131:36889), 65fabf990b834552aad823f7bc88a258 (127.16.224.129:44279)
I20260809 16:15:07.097898 15600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930b8b326ba849039ad8177307e069d2" candidate_uuid: "87c568b1793c4e86adf57c8c87cde0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fabf990b834552aad823f7bc88a258"
I20260809 16:15:07.097898 16000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930b8b326ba849039ad8177307e069d2" candidate_uuid: "87c568b1793c4e86adf57c8c87cde0be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670bb48be5874ae19564bb71949401bc"
I20260809 16:15:07.097986 15600 raft_consensus.cc:3037] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:07.097986 16000 raft_consensus.cc:3037] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:07.098909 15600 raft_consensus.cc:2454] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87c568b1793c4e86adf57c8c87cde0be in term 1.
I20260809 16:15:07.098909 16000 raft_consensus.cc:2454] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87c568b1793c4e86adf57c8c87cde0be in term 1.
I20260809 16:15:07.099073 15703 leader_election.cc:302] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 670bb48be5874ae19564bb71949401bc, 87c568b1793c4e86adf57c8c87cde0be; no voters:
I20260809 16:15:07.099164 16103 raft_consensus.cc:2781] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:07.099309 16103 raft_consensus.cc:686] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 1 LEADER]: Becoming Leader. State: Replica: 87c568b1793c4e86adf57c8c87cde0be, State: Running, Role: LEADER
I20260809 16:15:07.099391 16103 consensus_queue.cc:237] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } }
I20260809 16:15:07.100234 15379 catalog_manager.cc:5495] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be reported cstate change: term changed from 0 to 1, leader changed from <none> to 87c568b1793c4e86adf57c8c87cde0be (127.16.224.130). New cstate: current_term: 1 leader_uuid: "87c568b1793c4e86adf57c8c87cde0be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87c568b1793c4e86adf57c8c87cde0be" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 33517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 } health_report { overall_health: UNKNOWN } } }
W20260809 16:15:07.106860 15670 tablet.cc:2369] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:15:07.127776 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.125475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49352 (local address 127.16.224.130:33517)
0809 16:15:07.125538 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.125540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.125548 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:07.125587 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.125588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.125607 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.125666 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:07.125792 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.126270 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.127057 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.127263 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.127319 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.127471 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:07.127583 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:07.127656 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.127675 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.127695 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:15:07.129756 16000 raft_consensus.cc:1261] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 1 FOLLOWER]: Refusing update from remote peer 87c568b1793c4e86adf57c8c87cde0be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:07.129762 15600 raft_consensus.cc:1261] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 1 FOLLOWER]: Refusing update from remote peer 87c568b1793c4e86adf57c8c87cde0be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:07.130110 16110 consensus_queue.cc:1035] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [LEADER]: Connected to new peer: Peer: permanent_uuid: "670bb48be5874ae19564bb71949401bc" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:07.130285 16103 consensus_queue.cc:1035] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [LEADER]: Connected to new peer: Peer: permanent_uuid: "65fabf990b834552aad823f7bc88a258" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 44279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:07.132773 16119 mvcc.cc:205] Tried to move back new op lower bound from 7316652470800429056 to 7316652470679752704. Current Snapshot: MvccSnapshot[applied={T|T < 7316652470800429056}]
I20260809 16:15:07.132782 16117 mvcc.cc:205] Tried to move back new op lower bound from 7316652470800429056 to 7316652470679752704. Current Snapshot: MvccSnapshot[applied={T|T < 7316652470800429056}]
I20260809 16:15:07.134752 15380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:07.134857 15380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:07.135579 15380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a372d110243f4f279caf3722145acb6e: Sending DeleteTablet for 3 replicas of tablet 930b8b326ba849039ad8177307e069d2
I20260809 16:15:07.135942 15570 tablet_service.cc:1514] Processing DeleteTablet for tablet 930b8b326ba849039ad8177307e069d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:07 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:46458
I20260809 16:15:07.135943 15970 tablet_service.cc:1514] Processing DeleteTablet for tablet 930b8b326ba849039ad8177307e069d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:07 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:45408
I20260809 16:15:07.136226 16129 tablet_replica.cc:330] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258: stopping tablet replica
I20260809 16:15:07.136276 15770 tablet_service.cc:1514] Processing DeleteTablet for tablet 930b8b326ba849039ad8177307e069d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:07 UTC) from {username='kudu', principal='kudu/127.16.224.190@KRBTEST.COM'} at 127.0.0.1:49338
I20260809 16:15:07.136327 16129 raft_consensus.cc:2229] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:07.136423 16129 raft_consensus.cc:2258] T 930b8b326ba849039ad8177307e069d2 P 65fabf990b834552aad823f7bc88a258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:07.136447 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 15472
I20260809 16:15:07.144743 16130 tablet_replica.cc:330] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: stopping tablet replica
I20260809 16:15:07.144906 16130 raft_consensus.cc:2229] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:07.145043 16130 raft_consensus.cc:2258] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:07.147295 16130 ts_tablet_manager.cc:1905] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:07.147732 16131 tablet_replica.cc:330] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: stopping tablet replica
I20260809 16:15:07.147902 16131 raft_consensus.cc:2229] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:15:07.148063 16131 raft_consensus.cc:2258] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:07.148515 16131 ts_tablet_manager.cc:1905] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:07.148957 16130 ts_tablet_manager.cc:1918] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:07.149042 16130 log.cc:1196] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/930b8b326ba849039ad8177307e069d2
I20260809 16:15:07.149354 16130 ts_tablet_manager.cc:1939] T 930b8b326ba849039ad8177307e069d2 P 670bb48be5874ae19564bb71949401bc: Deleting consensus metadata
I20260809 16:15:07.149837 15362 catalog_manager.cc:4841] TS 670bb48be5874ae19564bb71949401bc (127.16.224.131:36889): tablet 930b8b326ba849039ad8177307e069d2 (table test-table [id=74900b229c03488eb616f76ab29822a1]) successfully deleted
I20260809 16:15:07.150563 16131 ts_tablet_manager.cc:1918] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:07.150688 16131 log.cc:1196] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/930b8b326ba849039ad8177307e069d2
I20260809 16:15:07.150976 16131 ts_tablet_manager.cc:1939] T 930b8b326ba849039ad8177307e069d2 P 87c568b1793c4e86adf57c8c87cde0be: Deleting consensus metadata
I20260809 16:15:07.151391 15361 catalog_manager.cc:4841] TS 87c568b1793c4e86adf57c8c87cde0be (127.16.224.130:33517): tablet 930b8b326ba849039ad8177307e069d2 (table test-table [id=74900b229c03488eb616f76ab29822a1]) successfully deleted
W20260809 16:15:07.152463 15364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:44279 (error 108)
I20260809 16:15:07.152710 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 15675
I20260809 16:15:07.154665 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.152831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:44279 (local address 127.0.0.1:46472)
0809 16:15:07.154590 (+ 1759us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.154635 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:44279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1734}
W20260809 16:15:07.154794 15364 catalog_manager.cc:4584] TS 65fabf990b834552aad823f7bc88a258 (127.16.224.129:44279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 930b8b326ba849039ad8177307e069d2: Network error: Client connection negotiation failed: client connection to 127.16.224.129:44279: connect: Connection refused (error 111)
I20260809 16:15:07.160766 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 15877
I20260809 16:15:07.168864 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 15335
2026-08-09T16:15:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2129 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T16:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:07Z Disabled control of system clock
I20260809 16:15:07.201781 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:39093
--webserver_interface=127.16.224.190
--webserver_port=0
--builtin_ntp_servers=127.16.224.148:43815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:39093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:07.284003 16138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:07.284220 16138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:07.284241 16138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:07.285328 16138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:07.285365 16138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:07.285378 16138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:07.285391 16138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:07.285410 16138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:07.286801 16138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:39093
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:39093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:07.287007 16138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:07.287263 16138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:07.291922 16143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:07.291982 16145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:07.292033 16138 server_base.cc:1034] running on GCE node
W20260809 16:15:07.291929 16147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:07.292450 16138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:07.292697 16138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:07.293859 16138 hybrid_clock.cc:648] HybridClock initialized: now 1786292107293838 us; error 31 us; skew 500 ppm
I20260809 16:15:07.295128 16138 webserver.cc:460] Webserver started at http://127.16.224.190:35283/ using document root <none> and password file <none>
I20260809 16:15:07.295373 16138 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:07.295428 16138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:07.295563 16138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:07.296662 16138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "4765068942584ee0a813d1e1603d3af9"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "f881e0f57400b2d0f5fb5eeb4bc747c3"
server_key_iv: "af7d72515e4341ccd9ee9619aed1212e"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.297045 16138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "4765068942584ee0a813d1e1603d3af9"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "f881e0f57400b2d0f5fb5eeb4bc747c3"
server_key_iv: "af7d72515e4341ccd9ee9619aed1212e"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.298424 16138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:07.299160 16155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.299314 16138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:07.299392 16138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "4765068942584ee0a813d1e1603d3af9"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "f881e0f57400b2d0f5fb5eeb4bc747c3"
server_key_iv: "af7d72515e4341ccd9ee9619aed1212e"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.299463 16138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:07.326681 16138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:07.327008 16138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:07.327136 16138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:07.331009 16138 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:39093
I20260809 16:15:07.331048 16246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:39093 every 8 connection(s)
I20260809 16:15:07.331369 16138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:07.332306 16247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:07.334609 16247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9: Bootstrap starting.
I20260809 16:15:07.335170 16247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:07.335440 16247 log.cc:824] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:07.336084 16247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9: No bootstrap required, opened a new log
I20260809 16:15:07.336293 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 16138
I20260809 16:15:07.336400 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:07.336575 17282 external_mini_cluster.cc:1428] Setting key d2abcadf5e2a98fadfd174c161ed6de9
I20260809 16:15:07.337857 16247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4765068942584ee0a813d1e1603d3af9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39093 } }
I20260809 16:15:07.338019 16247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:07.338054 16247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4765068942584ee0a813d1e1603d3af9, State: Initialized, Role: FOLLOWER
I20260809 16:15:07.338196 16247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4765068942584ee0a813d1e1603d3af9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39093 } }
I20260809 16:15:07.338289 16247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:07.338343 16247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:07.338395 16247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:07.339046 16247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4765068942584ee0a813d1e1603d3af9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39093 } }
I20260809 16:15:07.339159 16247 leader_election.cc:302] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4765068942584ee0a813d1e1603d3af9; no voters:
I20260809 16:15:07.339315 16247 leader_election.cc:288] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:07.339357 16254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:07.339506 16254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [term 1 LEADER]: Becoming Leader. State: Replica: 4765068942584ee0a813d1e1603d3af9, State: Running, Role: LEADER
I20260809 16:15:07.339542 16247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:07.339671 16254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4765068942584ee0a813d1e1603d3af9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39093 } }
I20260809 16:15:07.340190 16255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4765068942584ee0a813d1e1603d3af9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4765068942584ee0a813d1e1603d3af9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39093 } } }
I20260809 16:15:07.340197 16257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4765068942584ee0a813d1e1603d3af9. Latest consensus state: current_term: 1 leader_uuid: "4765068942584ee0a813d1e1603d3af9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4765068942584ee0a813d1e1603d3af9" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39093 } } }
I20260809 16:15:07.340284 16255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:07.340305 16257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:07.340591 16265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:07.341092 16265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:07.342692 16265 catalog_manager.cc:1347] Generated new cluster ID: 82d575c7902744e4a843eade724033b0
I20260809 16:15:07.342751 16265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:07.343683 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.337050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.16.224.190:39093)
0809 16:15:07.337259 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.337266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.337305 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:15:07.337831 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.337837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.337890 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.338020 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:07.338263 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.340555 (+ 2292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.341701 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.342248 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.342387 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.343057 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:07.343061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:07.343062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:07.343234 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:07.343268 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:07.343278 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:07.343333 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.343411 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.343492 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260809 16:15:07.356078 16265 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:07.356775 16265 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:07.359397 16265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9: Generated new TSK 0
I20260809 16:15:07.359570 16265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:07.366521 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.129:0
--local_ip_for_outbound_sockets=127.16.224.129
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39093
--builtin_ntp_servers=127.16.224.148:43815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:07.452178 16281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:07.452394 16281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:07.452432 16281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:07.453579 16281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:07.453617 16281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:07.453649 16281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.129
I20260809 16:15:07.455021 16281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.224.129
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.129
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:07.455246 16281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:07.455545 16281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:07.461364 16290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:07.461352 16291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:07.461344 16294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:07.461721 16281 server_base.cc:1034] running on GCE node
I20260809 16:15:07.461915 16281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:07.462157 16281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:07.463307 16281 hybrid_clock.cc:648] HybridClock initialized: now 1786292107463281 us; error 44 us; skew 500 ppm
I20260809 16:15:07.464615 16281 webserver.cc:460] Webserver started at http://127.16.224.129:45073/ using document root <none> and password file <none>
I20260809 16:15:07.464843 16281 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:07.464885 16281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:07.465010 16281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:07.466066 16281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data/instance:
uuid: "ee0651580030448f8ae5e25b88a4ad72"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "ca278493834371dee1aaa7bb7c8d6e79"
server_key_iv: "f6b79b440f86ec08ea0dbbdb173cad22"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.466432 16281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance:
uuid: "ee0651580030448f8ae5e25b88a4ad72"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "ca278493834371dee1aaa7bb7c8d6e79"
server_key_iv: "f6b79b440f86ec08ea0dbbdb173cad22"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.467635 16281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:15:07.468333 16306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.468474 16281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:07.468562 16281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/wal
uuid: "ee0651580030448f8ae5e25b88a4ad72"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "ca278493834371dee1aaa7bb7c8d6e79"
server_key_iv: "f6b79b440f86ec08ea0dbbdb173cad22"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.468632 16281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:07.487558 16281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:07.487881 16281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:07.488013 16281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:07.488232 16281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:07.488593 16281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:07.488631 16281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.488651 16281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:07.488663 16281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.494359 16281 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.129:38111
I20260809 16:15:07.494387 16482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.129:38111 every 8 connection(s)
I20260809 16:15:07.494752 16281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/data/info.pb
I20260809 16:15:07.499825 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.495487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.129:42185 (local address 127.16.224.190:39093)
0809 16:15:07.495547 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.495548 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.495771 (+ 223us) server_negotiation.cc:407] Connection header received
0809 16:15:07.496290 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.496292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.496308 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.496360 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:07.497042 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.497539 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.498569 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.498762 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.498816 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.499572 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:07.499573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:07.499573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:07.499578 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:07.499593 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:07.499597 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:07.499638 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.499716 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.499744 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:15:07.500036 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.495476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39093 (local address 127.16.224.129:42185)
0809 16:15:07.495658 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.495692 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.496192 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.496435 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.496439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.496449 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:07.496953 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.496958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.497628 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.497629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.498482 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.498484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.498829 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.498830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.498972 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.499446 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:07.499446 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:07.499475 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:07.499497 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:07.499645 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:07.499650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:07.499653 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.499714 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.499814 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260809 16:15:07.500485 16484 heartbeater.cc:347] Connected to a master server at 127.16.224.190:39093
I20260809 16:15:07.500563 16484 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:07.500734 16484 heartbeater.cc:511] Master 127.16.224.190:39093 requested a full tablet report, sending...
I20260809 16:15:07.501194 16187 ts_manager.cc:194] Registered new tserver with Master: ee0651580030448f8ae5e25b88a4ad72 (127.16.224.129:38111)
I20260809 16:15:07.501791 16187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.129:42185
I20260809 16:15:07.503543 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 16281
I20260809 16:15:07.503633 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-0/wal/instance
I20260809 16:15:07.503826 17282 external_mini_cluster.cc:1428] Setting key e00daeb9a9695bf4cb808d9156a74453
I20260809 16:15:07.504676 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.130:0
--local_ip_for_outbound_sockets=127.16.224.130
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39093
--builtin_ntp_servers=127.16.224.148:43815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:15:07.585807 16488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:07.586002 16488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:07.586021 16488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:07.587178 16488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:07.587222 16488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:07.587245 16488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.130
I20260809 16:15:07.588532 16488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.224.130
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.130
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:07.588711 16488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:07.588956 16488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:07.593684 16499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:07.593684 16494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:07.593710 16496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:07.593858 16488 server_base.cc:1034] running on GCE node
I20260809 16:15:07.594099 16488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:07.594357 16488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:07.595513 16488 hybrid_clock.cc:648] HybridClock initialized: now 1786292107595499 us; error 31 us; skew 500 ppm
I20260809 16:15:07.596726 16488 webserver.cc:460] Webserver started at http://127.16.224.130:35129/ using document root <none> and password file <none>
I20260809 16:15:07.596952 16488 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:07.597002 16488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:07.597128 16488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:07.598136 16488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data/instance:
uuid: "15c351bb39be4783b99b5c98d29be1cf"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "d3fce60b151d6f2a31eef934f91ce16a"
server_key_iv: "828fae7c78fa20bcced7527d9933cade"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.598501 16488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance:
uuid: "15c351bb39be4783b99b5c98d29be1cf"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "d3fce60b151d6f2a31eef934f91ce16a"
server_key_iv: "828fae7c78fa20bcced7527d9933cade"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.599677 16488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:15:07.600348 16513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.600497 16488 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:07.600570 16488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal
uuid: "15c351bb39be4783b99b5c98d29be1cf"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "d3fce60b151d6f2a31eef934f91ce16a"
server_key_iv: "828fae7c78fa20bcced7527d9933cade"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.600638 16488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:07.609737 16488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:07.609975 16488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:07.610090 16488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:07.610276 16488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:07.610606 16488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:07.610641 16488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.610667 16488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:07.610684 16488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.616042 16488 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.130:38975
I20260809 16:15:07.616087 16689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.130:38975 every 8 connection(s)
I20260809 16:15:07.616379 16488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/data/info.pb
I20260809 16:15:07.619817 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 16488
I20260809 16:15:07.619926 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal/instance
I20260809 16:15:07.620138 17282 external_mini_cluster.cc:1428] Setting key f9d6cc213f3745001bc4d31ed336cb40
I20260809 16:15:07.621093 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.224.131:0
--local_ip_for_outbound_sockets=127.16.224.131
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39093
--builtin_ntp_servers=127.16.224.148:43815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:15:07.621629 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.617089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.130:33815 (local address 127.16.224.190:39093)
0809 16:15:07.617157 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.617159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.617377 (+ 218us) server_negotiation.cc:407] Connection header received
0809 16:15:07.617835 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.617836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.617846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.617894 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:07.618569 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.619057 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.620273 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.620512 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.620574 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.621373 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:07.621374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:07.621375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:07.621379 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:07.621394 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:07.621399 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:07.621444 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.621521 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.621550 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:15:07.621825 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.617069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39093 (local address 127.16.224.130:33815)
0809 16:15:07.617266 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.617298 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.617755 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.617948 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.617951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.617959 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:07.618499 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.618504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.619130 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.619131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.620173 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.620176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.620596 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.620597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.620720 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.621239 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:07.621240 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:07.621268 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:07.621290 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:07.621455 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:07.621461 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:07.621464 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.621523 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.621621 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260809 16:15:07.622283 16691 heartbeater.cc:347] Connected to a master server at 127.16.224.190:39093
I20260809 16:15:07.622419 16691 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:07.622608 16691 heartbeater.cc:511] Master 127.16.224.190:39093 requested a full tablet report, sending...
I20260809 16:15:07.622994 16187 ts_manager.cc:194] Registered new tserver with Master: 15c351bb39be4783b99b5c98d29be1cf (127.16.224.130:38975)
I20260809 16:15:07.623353 16187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.130:33815
W20260809 16:15:07.702199 16695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:07.702435 16695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:07.702458 16695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:07.703629 16695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:07.703668 16695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:07.703692 16695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.224.131
I20260809 16:15:07.704998 16695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.224.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.224.131
--webserver_port=0
--tserver_master_addrs=127.16.224.190:39093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.224.131
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:07.705204 16695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:07.705482 16695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:15:07.710392 16695 server_base.cc:1034] running on GCE node
W20260809 16:15:07.710274 16706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:07.710271 16712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:07.710304 16704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:07.710767 16695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:07.711015 16695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:07.712149 16695 hybrid_clock.cc:648] HybridClock initialized: now 1786292107712132 us; error 30 us; skew 500 ppm
I20260809 16:15:07.713440 16695 webserver.cc:460] Webserver started at http://127.16.224.131:34469/ using document root <none> and password file <none>
I20260809 16:15:07.713652 16695 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:07.713701 16695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:07.713829 16695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:07.714920 16695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data/instance:
uuid: "04339123dc844ba5bb54f3d9a275b6da"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "c1935c48d693ff53c0753decfd8a2a00"
server_key_iv: "da1274f80375916ddfd7bf2ac81ef500"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.715253 16695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance:
uuid: "04339123dc844ba5bb54f3d9a275b6da"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "c1935c48d693ff53c0753decfd8a2a00"
server_key_iv: "da1274f80375916ddfd7bf2ac81ef500"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.716461 16695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:07.717095 16721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.717249 16695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:07.717334 16695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal
uuid: "04339123dc844ba5bb54f3d9a275b6da"
format_stamp: "Formatted at 2026-08-09 16:15:07 on dist-test-slave-fgb3"
server_key: "c1935c48d693ff53c0753decfd8a2a00"
server_key_iv: "da1274f80375916ddfd7bf2ac81ef500"
server_key_version: "encryptionkey@0"
I20260809 16:15:07.717402 16695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:07.734433 16695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:07.734727 16695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:07.734858 16695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:07.735069 16695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:15:07.735394 16695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:15:07.735430 16695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.735459 16695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:15:07.735482 16695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:07.741278 16695 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.131:43437
I20260809 16:15:07.741344 16892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.131:43437 every 8 connection(s)
I20260809 16:15:07.741688 16695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/data/info.pb
I20260809 16:15:07.746876 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.742420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:59403 (local address 127.16.224.190:39093)
0809 16:15:07.742492 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.742494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.742757 (+ 263us) server_negotiation.cc:407] Connection header received
0809 16:15:07.743238 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.743239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.743248 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.743296 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:07.744006 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.744496 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.745525 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.745721 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.745785 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.746597 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:07.746599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:07.746599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:07.746603 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:07.746619 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:07.746623 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:07.746667 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.746786 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.746816 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:15:07.746861 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 16695
I20260809 16:15:07.746968 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal/instance
I20260809 16:15:07.747116 16895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.742420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39093 (local address 127.16.224.131:59403)
0809 16:15:07.742621 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.742667 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.743156 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.743376 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.743380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.743388 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:07.743931 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.743936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.744564 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.744565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.745421 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.745424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.745793 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.745793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.745907 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.746423 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:15:07.746423 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:07.746452 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:15:07.746496 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:07.746678 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:07.746682 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:07.746685 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.746790 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.746900 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260809 16:15:07.747196 17282 external_mini_cluster.cc:1428] Setting key ebb97662fcb9d579ea5f17c6d7a0002a
I20260809 16:15:07.747560 16893 heartbeater.cc:347] Connected to a master server at 127.16.224.190:39093
I20260809 16:15:07.747666 16893 heartbeater.cc:464] Registering TS with master...
I20260809 16:15:07.747864 16893 heartbeater.cc:511] Master 127.16.224.190:39093 requested a full tablet report, sending...
I20260809 16:15:07.748219 16187 ts_manager.cc:194] Registered new tserver with Master: 04339123dc844ba5bb54f3d9a275b6da (127.16.224.131:43437)
I20260809 16:15:07.748564 16187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.224.131:59403
I20260809 16:15:07.758808 17282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:15:07.762887 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.760179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35846 (local address 127.16.224.190:39093)
0809 16:15:07.760245 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.760247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.760343 (+ 96us) server_negotiation.cc:407] Connection header received
0809 16:15:07.760403 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.760404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.760416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.760458 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:07.760602 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.761073 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.761789 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.761981 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.762032 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.762661 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:07.762662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:07.762663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:07.762667 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:07.762681 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:07.762684 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:07.762724 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.762777 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.762807 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:15:07.764261 16187 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:15:07.765861 16187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:15:07.772357 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.768487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:38111 (local address 127.0.0.1:45528)
0809 16:15:07.768838 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.768847 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.768887 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.769139 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.769140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.769142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:07.769176 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.769177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.770358 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.770359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.771296 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.771299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.772037 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.772039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.772123 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.772138 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.772213 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.772271 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":80,"threads_started":1}
I20260809 16:15:07.772495 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.768291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.131:43437 (local address 127.0.0.1:46938)
0809 16:15:07.768457 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.768468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.768523 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.768829 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.768833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.768835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:07.768912 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.768915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.770271 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.770272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.771385 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.771388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.772206 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.772207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.772274 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.772277 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.772343 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.772377 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":30,"thread_start_us":97,"threads_started":1}
I20260809 16:15:07.772655 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.768388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46938 (local address 127.16.224.131:43437)
0809 16:15:07.768638 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.768642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.768652 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:15:07.768692 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.768694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.768711 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.768990 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:07.769048 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.770180 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.771454 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.772133 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.772208 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.772292 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.772358 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.772389 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
I20260809 16:15:07.772794 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.768621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45528 (local address 127.16.224.129:38111)
0809 16:15:07.768999 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.769003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.769011 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:07.769047 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.769048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.769062 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.769136 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:07.769237 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.770267 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.771378 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.771941 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.772020 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.772120 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.772638 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.772690 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":110,"threads_started":1}
I20260809 16:15:07.772954 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.768185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.16.224.130:38975)
0809 16:15:07.768678 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:07.768682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:07.768692 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:15:07.768734 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:07.768735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:07.768764 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:07.768844 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:07.769059 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.769843 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.771295 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.772133 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.772208 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.772292 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:07.772830 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 16:15:07.772877 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":101,"threads_started":1}
I20260809 16:15:07.772958 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:07.768054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:38975 (local address 127.0.0.1:46054)
0809 16:15:07.768206 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:07.768271 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:07.768341 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:07.768923 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:07.768925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:07.768927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:07.768973 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.768975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.769932 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.769933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.771200 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:07.771203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:07.772409 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:07.772409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:07.772641 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:07.772646 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:15:07.772792 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 16:15:07.772833 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260809 16:15:07.773363 16788 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2b39f0a5a545daafaff25223bde98f (DEFAULT_TABLE table=test-table [id=d485471b3de545518583b10e5826ca1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:07.773460 16381 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2b39f0a5a545daafaff25223bde98f (DEFAULT_TABLE table=test-table [id=d485471b3de545518583b10e5826ca1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:07.773693 16788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2b39f0a5a545daafaff25223bde98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:07.773830 16381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2b39f0a5a545daafaff25223bde98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:07.776399 16921 tablet_bootstrap.cc:492] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Bootstrap starting.
I20260809 16:15:07.776641 16923 tablet_bootstrap.cc:492] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: Bootstrap starting.
I20260809 16:15:07.777171 16921 tablet_bootstrap.cc:654] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:07.777264 16923 tablet_bootstrap.cc:654] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:07.777529 16921 log.cc:824] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:07.777549 16923 log.cc:824] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:07.777601 16591 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2b39f0a5a545daafaff25223bde98f (DEFAULT_TABLE table=test-table [id=d485471b3de545518583b10e5826ca1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:15:07.778023 16591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2b39f0a5a545daafaff25223bde98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:07.778369 16921 tablet_bootstrap.cc:492] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: No bootstrap required, opened a new log
I20260809 16:15:07.778472 16923 tablet_bootstrap.cc:492] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: No bootstrap required, opened a new log
I20260809 16:15:07.778481 16921 ts_tablet_manager.cc:1397] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:15:07.778541 16923 ts_tablet_manager.cc:1397] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:07.780470 16923 raft_consensus.cc:348] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:07.780609 16923 raft_consensus.cc:374] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:07.780642 16923 raft_consensus.cc:729] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0651580030448f8ae5e25b88a4ad72, State: Initialized, Role: FOLLOWER
I20260809 16:15:07.780764 16923 consensus_queue.cc:260] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:07.781029 16923 ts_tablet_manager.cc:1428] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:15:07.781018 16921 raft_consensus.cc:348] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:07.781172 16484 heartbeater.cc:503] Master 127.16.224.190:39093 was elected leader, sending a full tablet report...
I20260809 16:15:07.781200 16921 raft_consensus.cc:374] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:07.781260 16921 raft_consensus.cc:729] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04339123dc844ba5bb54f3d9a275b6da, State: Initialized, Role: FOLLOWER
I20260809 16:15:07.781383 16921 consensus_queue.cc:260] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:07.781641 16921 ts_tablet_manager.cc:1428] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:15:07.781672 16893 heartbeater.cc:503] Master 127.16.224.190:39093 was elected leader, sending a full tablet report...
I20260809 16:15:07.781906 16930 tablet_bootstrap.cc:492] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Bootstrap starting.
I20260809 16:15:07.782634 16930 tablet_bootstrap.cc:654] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:07.782907 16930 log.cc:824] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:07.783537 16930 tablet_bootstrap.cc:492] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: No bootstrap required, opened a new log
I20260809 16:15:07.783597 16930 ts_tablet_manager.cc:1397] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:15:07.785432 16930 raft_consensus.cc:348] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:07.785561 16930 raft_consensus.cc:374] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:07.785583 16930 raft_consensus.cc:729] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c351bb39be4783b99b5c98d29be1cf, State: Initialized, Role: FOLLOWER
I20260809 16:15:07.785667 16930 consensus_queue.cc:260] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:07.785899 16930 ts_tablet_manager.cc:1428] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:15:07.785969 16691 heartbeater.cc:503] Master 127.16.224.190:39093 was elected leader, sending a full tablet report...
W20260809 16:15:07.866928 16692 tablet.cc:2369] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:15:07.992286 16894 tablet.cc:2369] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:15:07.995420 16485 tablet.cc:2369] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:15:08.044536 16932 raft_consensus.cc:482] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:15:08.044679 16932 raft_consensus.cc:504] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:08.045006 16932 leader_election.cc:288] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c351bb39be4783b99b5c98d29be1cf (127.16.224.130:38975), ee0651580030448f8ae5e25b88a4ad72 (127.16.224.129:38111)
I20260809 16:15:08.048801 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:08.045277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:56857 (local address 127.16.224.130:38975)
0809 16:15:08.045327 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:08.045329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:08.045343 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:15:08.045413 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:08.045414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:08.045424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:08.045483 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:08.045656 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.046448 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.047898 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.048448 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.048522 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:08.048531 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:08.048692 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:15:08.048728 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:15:08.048801 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:08.045403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.224.131:43063 (local address 127.16.224.129:38111)
0809 16:15:08.045463 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:08.045465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:08.045536 (+ 71us) server_negotiation.cc:407] Connection header received
0809 16:15:08.045575 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:08.045576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:08.045584 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:08.045631 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:15:08.045752 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.046468 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.047897 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.048456 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.048522 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:08.048531 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:08.048692 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:15:08.048728 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:15:08.048827 16895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:08.045189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.130:38975 (local address 127.16.224.131:56857)
0809 16:15:08.045247 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:08.045259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:08.045319 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:08.045502 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:08.045503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:08.045505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:08.045587 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:08.045589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.046540 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.046541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:08.047814 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:08.047818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.048528 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.048529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:08.048620 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:08.048624 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:15:08.048678 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:15:08.048723 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":50}
I20260809 16:15:08.048832 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:08.045297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:38111 (local address 127.16.224.131:43063)
0809 16:15:08.045469 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:08.045479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:08.045529 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:08.045645 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:08.045645 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:08.045647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:15:08.045696 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:08.045697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.046552 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.046553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:08.047814 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:08.047818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.048528 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.048529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:08.048624 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:08.048626 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:15:08.048678 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:15:08.048719 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":19,"thread_start_us":112,"threads_started":1}
I20260809 16:15:08.049203 16620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2b39f0a5a545daafaff25223bde98f" candidate_uuid: "04339123dc844ba5bb54f3d9a275b6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c351bb39be4783b99b5c98d29be1cf" is_pre_election: true
I20260809 16:15:08.049239 16416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2b39f0a5a545daafaff25223bde98f" candidate_uuid: "04339123dc844ba5bb54f3d9a275b6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0651580030448f8ae5e25b88a4ad72" is_pre_election: true
I20260809 16:15:08.049357 16620 raft_consensus.cc:2454] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04339123dc844ba5bb54f3d9a275b6da in term 0.
I20260809 16:15:08.049386 16416 raft_consensus.cc:2454] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04339123dc844ba5bb54f3d9a275b6da in term 0.
I20260809 16:15:08.049569 16728 leader_election.cc:302] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04339123dc844ba5bb54f3d9a275b6da, 15c351bb39be4783b99b5c98d29be1cf; no voters:
I20260809 16:15:08.049696 16932 raft_consensus.cc:2781] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:15:08.049755 16932 raft_consensus.cc:482] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:15:08.049786 16932 raft_consensus.cc:3037] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:08.050511 16932 raft_consensus.cc:504] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:08.050619 16932 leader_election.cc:288] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [CANDIDATE]: Term 1 election: Requested vote from peers 15c351bb39be4783b99b5c98d29be1cf (127.16.224.130:38975), ee0651580030448f8ae5e25b88a4ad72 (127.16.224.129:38111)
I20260809 16:15:08.050797 16620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2b39f0a5a545daafaff25223bde98f" candidate_uuid: "04339123dc844ba5bb54f3d9a275b6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c351bb39be4783b99b5c98d29be1cf"
I20260809 16:15:08.050813 16416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2b39f0a5a545daafaff25223bde98f" candidate_uuid: "04339123dc844ba5bb54f3d9a275b6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0651580030448f8ae5e25b88a4ad72"
I20260809 16:15:08.050874 16620 raft_consensus.cc:3037] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:08.050882 16416 raft_consensus.cc:3037] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:08.051561 16620 raft_consensus.cc:2454] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04339123dc844ba5bb54f3d9a275b6da in term 1.
I20260809 16:15:08.051635 16416 raft_consensus.cc:2454] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04339123dc844ba5bb54f3d9a275b6da in term 1.
I20260809 16:15:08.051744 16728 leader_election.cc:302] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04339123dc844ba5bb54f3d9a275b6da, 15c351bb39be4783b99b5c98d29be1cf; no voters:
I20260809 16:15:08.051836 16932 raft_consensus.cc:2781] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:08.051964 16932 raft_consensus.cc:686] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 1 LEADER]: Becoming Leader. State: Replica: 04339123dc844ba5bb54f3d9a275b6da, State: Running, Role: LEADER
I20260809 16:15:08.052050 16932 consensus_queue.cc:237] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } }
I20260809 16:15:08.052840 16187 catalog_manager.cc:5495] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da reported cstate change: term changed from 0 to 1, leader changed from <none> to 04339123dc844ba5bb54f3d9a275b6da (127.16.224.131). New cstate: current_term: 1 leader_uuid: "04339123dc844ba5bb54f3d9a275b6da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04339123dc844ba5bb54f3d9a275b6da" member_type: VOTER last_known_addr { host: "127.16.224.131" port: 43437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 } health_report { overall_health: UNKNOWN } } }
I20260809 16:15:08.123701 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:08.121424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.16.224.131:43437)
0809 16:15:08.121499 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:08.121501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:08.121511 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:15:08.121557 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:08.121558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:08.121579 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:08.121650 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:08.121747 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.122214 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.122951 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:08.123126 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:08.123173 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:08.123352 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:08.123456 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:08.123518 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:08.123597 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 16:15:08.123612 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:15:08.125519 16416 raft_consensus.cc:1261] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72 [term 1 FOLLOWER]: Refusing update from remote peer 04339123dc844ba5bb54f3d9a275b6da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:08.125519 16620 raft_consensus.cc:1261] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 1 FOLLOWER]: Refusing update from remote peer 04339123dc844ba5bb54f3d9a275b6da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:08.125838 16932 consensus_queue.cc:1035] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee0651580030448f8ae5e25b88a4ad72" member_type: VOTER last_known_addr { host: "127.16.224.129" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:08.125986 16936 consensus_queue.cc:1035] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c351bb39be4783b99b5c98d29be1cf" member_type: VOTER last_known_addr { host: "127.16.224.130" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:08.128664 16943 mvcc.cc:205] Tried to move back new op lower bound from 7316652474879598592 to 7316652474581749760. Current Snapshot: MvccSnapshot[applied={T|T < 7316652474879598592}]
I20260809 16:15:08.128942 16946 mvcc.cc:205] Tried to move back new op lower bound from 7316652474879598592 to 7316652474581749760. Current Snapshot: MvccSnapshot[applied={T|T < 7316652474879598592}]
I20260809 16:15:08.130477 16187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:08.130573 16187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:15:08.131287 16187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4765068942584ee0a813d1e1603d3af9: Sending DeleteTablet for 3 replicas of tablet 3f2b39f0a5a545daafaff25223bde98f
I20260809 16:15:08.131601 16591 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2b39f0a5a545daafaff25223bde98f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:08 UTC) from {username='slave'} at 127.0.0.1:46054
I20260809 16:15:08.131655 16788 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2b39f0a5a545daafaff25223bde98f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:08 UTC) from {username='slave'} at 127.0.0.1:46938
I20260809 16:15:08.131687 16381 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2b39f0a5a545daafaff25223bde98f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:15:08 UTC) from {username='slave'} at 127.0.0.1:45528
I20260809 16:15:08.131892 16959 tablet_replica.cc:330] T 3f2b39f0a5a545daafaff25223bde98f P ee0651580030448f8ae5e25b88a4ad72: stopping tablet replica
I20260809 16:15:08.131930 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 16281
I20260809 16:15:08.132731 16961 tablet_replica.cc:330] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: stopping tablet replica
I20260809 16:15:08.132864 16961 raft_consensus.cc:2229] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:15:08.133010 16961 raft_consensus.cc:2258] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:08.140120 16958 tablet_replica.cc:330] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: stopping tablet replica
I20260809 16:15:08.140247 16961 ts_tablet_manager.cc:1905] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:08.140259 16958 raft_consensus.cc:2229] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:15:08.140409 16958 raft_consensus.cc:2258] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:15:08.142026 16961 ts_tablet_manager.cc:1918] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:08.142110 16961 log.cc:1196] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-2/wal/wals/3f2b39f0a5a545daafaff25223bde98f
I20260809 16:15:08.142360 16958 ts_tablet_manager.cc:1905] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:15:08.142446 16961 ts_tablet_manager.cc:1939] T 3f2b39f0a5a545daafaff25223bde98f P 04339123dc844ba5bb54f3d9a275b6da: Deleting consensus metadata
I20260809 16:15:08.142915 16162 catalog_manager.cc:4841] TS 04339123dc844ba5bb54f3d9a275b6da (127.16.224.131:43437): tablet 3f2b39f0a5a545daafaff25223bde98f (table test-table [id=d485471b3de545518583b10e5826ca1a]) successfully deleted
I20260809 16:15:08.143781 16958 ts_tablet_manager.cc:1918] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:15:08.143844 16958 log.cc:1196] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Deleting WAL directory at /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786291868496945-17282-0/minicluster-data/ts-1/wal/wals/3f2b39f0a5a545daafaff25223bde98f
I20260809 16:15:08.144059 16958 ts_tablet_manager.cc:1939] T 3f2b39f0a5a545daafaff25223bde98f P 15c351bb39be4783b99b5c98d29be1cf: Deleting consensus metadata
I20260809 16:15:08.144440 16162 catalog_manager.cc:4841] TS 15c351bb39be4783b99b5c98d29be1cf (127.16.224.130:38975): tablet 3f2b39f0a5a545daafaff25223bde98f (table test-table [id=d485471b3de545518583b10e5826ca1a]) successfully deleted
W20260809 16:15:08.147528 16162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.224.129:38111 (error 108)
I20260809 16:15:08.147729 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 16488
I20260809 16:15:08.149489 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:08.147846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.129:38111 (local address 127.0.0.1:45536)
0809 16:15:08.149399 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:08.149455 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.129:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1524}
W20260809 16:15:08.155695 16162 catalog_manager.cc:4584] TS ee0651580030448f8ae5e25b88a4ad72 (127.16.224.129:38111): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f2b39f0a5a545daafaff25223bde98f: Network error: Client connection negotiation failed: client connection to 127.16.224.129:38111: connect: Connection refused (error 111)
I20260809 16:15:08.165791 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 16695
I20260809 16:15:08.179176 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 16138
2026-08-09T16:15:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1011 ms)
[----------] 6 tests from EncryptionPolicyTest (9137 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:08 dist-test-slave-fgb3 krb5kdc[16968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:08 dist-test-slave-fgb3 krb5kdc[16968](info): set up 2 sockets
Aug 09 16:15:08 dist-test-slave-fgb3 krb5kdc[16968](info): commencing operation
krb5kdc: starting...
W20260809 16:15:09.332697 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:09 dist-test-slave-fgb3 krb5kdc[16968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:09.452672 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:09.535833 16989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:09.536036 16989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:09.536057 16989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:09.537176 16989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:09.537218 16989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:09.537230 16989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:09.537241 16989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:09.537261 16989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:09.539058 16989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:09.539319 16989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:09.539536 16989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:09.544355 17005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:09.544369 17000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:09.544476 16989 server_base.cc:1034] running on GCE node
W20260809 16:15:09.544390 17001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:09.544771 16989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:09.545063 16989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:09.546201 16989 hybrid_clock.cc:648] HybridClock initialized: now 1786292109546190 us; error 33 us; skew 500 ppm
Aug 09 16:15:09 dist-test-slave-fgb3 krb5kdc[16968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292109, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:09.552763 16989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:09.553279 16989 webserver.cc:460] Webserver started at http://127.0.0.1:34975/ using document root <none> and password file <none>
I20260809 16:15:09.553503 16989 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:09.553553 16989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:09.553682 16989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:09.554807 16989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "256d7d9b36d8460189cf5958c3dcf0a2"
format_stamp: "Formatted at 2026-08-09 16:15:09 on dist-test-slave-fgb3"
server_key: "a372994728b849ca845e056fd9b4e8d3"
server_key_iv: "b8516d7835aba379d5df998b236099fc"
server_key_version: "encryptionkey@0"
I20260809 16:15:09.555150 16989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "256d7d9b36d8460189cf5958c3dcf0a2"
format_stamp: "Formatted at 2026-08-09 16:15:09 on dist-test-slave-fgb3"
server_key: "a372994728b849ca845e056fd9b4e8d3"
server_key_iv: "b8516d7835aba379d5df998b236099fc"
server_key_version: "encryptionkey@0"
I20260809 16:15:09.556500 16989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:09.557299 17013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:09.557492 16989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:15:09.557581 16989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "256d7d9b36d8460189cf5958c3dcf0a2"
format_stamp: "Formatted at 2026-08-09 16:15:09 on dist-test-slave-fgb3"
server_key: "a372994728b849ca845e056fd9b4e8d3"
server_key_iv: "b8516d7835aba379d5df998b236099fc"
server_key_version: "encryptionkey@0"
I20260809 16:15:09.557643 16989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:09.570079 16989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:09.572032 16989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:09.572170 16989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:09.576213 16989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38363
I20260809 16:15:09.576263 17093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38363 every 8 connection(s)
I20260809 16:15:09.576602 16989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:09.577474 17094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:09.578203 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 16989
I20260809 16:15:09.578308 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:09.578514 17282 external_mini_cluster.cc:1428] Setting key 8958b36d029263e0ae742f45f39ec2f9
I20260809 16:15:09.580536 17094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2: Bootstrap starting.
I20260809 16:15:09.581130 17094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:09.581413 17094 log.cc:824] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:09.582141 17094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2: No bootstrap required, opened a new log
I20260809 16:15:09.584558 17094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260809 16:15:09.584715 17094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:09.584738 17094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256d7d9b36d8460189cf5958c3dcf0a2, State: Initialized, Role: FOLLOWER
I20260809 16:15:09.584857 17094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260809 16:15:09.584933 17094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:09.584956 17094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:09.585003 17094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 16:15:09 dist-test-slave-fgb3 krb5kdc[16968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:15:09.585577 17094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260809 16:15:09.585672 17094 leader_election.cc:302] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 256d7d9b36d8460189cf5958c3dcf0a2; no voters:
I20260809 16:15:09.585839 17094 leader_election.cc:288] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:09.585883 17104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:09.586041 17104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [term 1 LEADER]: Becoming Leader. State: Replica: 256d7d9b36d8460189cf5958c3dcf0a2, State: Running, Role: LEADER
I20260809 16:15:09.586162 17094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:09.586184 17104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260809 16:15:09.586604 17104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 256d7d9b36d8460189cf5958c3dcf0a2. Latest consensus state: current_term: 1 leader_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } } }
I20260809 16:15:09.586771 17104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:09.586992 17120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:09.587563 17120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:09.588608 17109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "256d7d9b36d8460189cf5958c3dcf0a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } } }
I20260809 16:15:09.588685 17109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:09.589818 17120 catalog_manager.cc:1347] Generated new cluster ID: 98da393f7b334635904b3b4179da27f8
I20260809 16:15:09.589865 17120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:09.590428 17100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:09.579035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43097 (local address 127.0.0.1:38363)
0809 16:15:09.579196 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:09.579204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:09.579229 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:15:09.579644 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:09.579651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:09.579697 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:09.579807 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:09.580581 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:09.581513 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:09.582509 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:09.582924 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:09.583011 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:09.585605 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:09.585609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:09.585609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:09.585622 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:09.588563 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:09.589032 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:09.589036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:09.589036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:09.589147 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:09.589390 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:09.589391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:09.589392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:09.589697 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:09.589833 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:09.590115 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 16:15:09.590217 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260809 16:15:09.591377 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:09.599723 17100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:09.592786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44115 (local address 127.0.0.1:38363)
0809 16:15:09.592826 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:09.592828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:09.592994 (+ 166us) server_negotiation.cc:407] Connection header received
0809 16:15:09.593053 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:09.593054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:09.593063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:09.593117 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:09.593721 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:09.594460 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:09.595440 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:09.595762 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:09.595819 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:09.597105 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:09.597105 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:09.597106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:09.597110 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:09.598572 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:09.598994 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:09.598995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:09.598995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:09.599044 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:09.599245 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:09.599246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:09.599246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:09.599362 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:09.599457 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:09.599582 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:15:09.599640 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:15:09.600623 17120 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:09.601086 17120 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:09.607275 17120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 256d7d9b36d8460189cf5958c3dcf0a2: Generated new TSK 0
I20260809 16:15:09.607451 17120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:09.611025 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 16989
W20260809 16:15:09.626561 16964 connection.cc:535] client connection to 127.0.0.1:38363 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38363): Connection reset by peer (error 104)
2026-08-09T16:15:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:09 dist-test-slave-fgb3 krb5kdc[17146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:09 dist-test-slave-fgb3 krb5kdc[17146](info): set up 2 sockets
Aug 09 16:15:09 dist-test-slave-fgb3 krb5kdc[17146](info): commencing operation
krb5kdc: starting...
W20260809 16:15:10.768574 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:10 dist-test-slave-fgb3 krb5kdc[17146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:10.886638 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38991
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:10.968420 17179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:10.968643 17179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:10.968675 17179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:10.969851 17179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:10.969895 17179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:10.969914 17179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:10.969929 17179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:10.969944 17179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:10.971496 17179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38991
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:10.971743 17179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:10.971971 17179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:10.976697 17192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:10.976787 17186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:10.976672 17188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:10.976899 17179 server_base.cc:1034] running on GCE node
I20260809 16:15:10.977128 17179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:10.977411 17179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:10.978564 17179 hybrid_clock.cc:648] HybridClock initialized: now 1786292110978548 us; error 29 us; skew 500 ppm
Aug 09 16:15:10 dist-test-slave-fgb3 krb5kdc[17146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:10.980994 17179 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:10.981431 17179 webserver.cc:460] Webserver started at http://127.0.0.1:36989/ using document root <none> and password file <none>
I20260809 16:15:10.981654 17179 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:10.981703 17179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:10.981825 17179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:10.982951 17179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "cfd414c3362447ba9037167ba3f86728"
format_stamp: "Formatted at 2026-08-09 16:15:10 on dist-test-slave-fgb3"
server_key: "e64b70603ed9fa68afa6f906137b98ab"
server_key_iv: "087dc7d00296e67bfb790fbfd1886bed"
server_key_version: "encryptionkey@0"
I20260809 16:15:10.983299 17179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "cfd414c3362447ba9037167ba3f86728"
format_stamp: "Formatted at 2026-08-09 16:15:10 on dist-test-slave-fgb3"
server_key: "e64b70603ed9fa68afa6f906137b98ab"
server_key_iv: "087dc7d00296e67bfb790fbfd1886bed"
server_key_version: "encryptionkey@0"
I20260809 16:15:10.984686 17179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:10.985380 17203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:10.985539 17179 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:10.985637 17179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "cfd414c3362447ba9037167ba3f86728"
format_stamp: "Formatted at 2026-08-09 16:15:10 on dist-test-slave-fgb3"
server_key: "e64b70603ed9fa68afa6f906137b98ab"
server_key_iv: "087dc7d00296e67bfb790fbfd1886bed"
server_key_version: "encryptionkey@0"
I20260809 16:15:10.985703 17179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:11.001389 17179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:11.003429 17179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:11.003571 17179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:11.007700 17179 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38991
I20260809 16:15:11.007737 17294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38991 every 8 connection(s)
I20260809 16:15:11.008086 17179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:11.008962 17296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:11.011340 17296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728: Bootstrap starting.
I20260809 16:15:11.011932 17296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:11.011984 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17179
I20260809 16:15:11.012091 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:11.012234 17296 log.cc:824] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:11.012293 17282 external_mini_cluster.cc:1428] Setting key cc615a4a14f3d042858cd32c3951b281
I20260809 16:15:11.013027 17296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728: No bootstrap required, opened a new log
I20260809 16:15:11.015654 17296 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd414c3362447ba9037167ba3f86728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260809 16:15:11.015800 17296 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:11.015822 17296 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd414c3362447ba9037167ba3f86728, State: Initialized, Role: FOLLOWER
I20260809 16:15:11.015945 17296 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd414c3362447ba9037167ba3f86728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260809 16:15:11.016016 17296 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:11.016039 17296 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:11.016084 17296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:11.016659 17296 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd414c3362447ba9037167ba3f86728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260809 16:15:11.016769 17296 leader_election.cc:302] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfd414c3362447ba9037167ba3f86728; no voters:
I20260809 16:15:11.016938 17296 leader_election.cc:288] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:11.016973 17303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:11.017129 17303 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [term 1 LEADER]: Becoming Leader. State: Replica: cfd414c3362447ba9037167ba3f86728, State: Running, Role: LEADER
I20260809 16:15:11.017210 17296 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:11.017268 17303 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd414c3362447ba9037167ba3f86728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } }
I20260809 16:15:11.017716 17306 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfd414c3362447ba9037167ba3f86728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd414c3362447ba9037167ba3f86728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } } }
I20260809 16:15:11.017797 17306 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:11.017979 17307 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd414c3362447ba9037167ba3f86728. Latest consensus state: current_term: 1 leader_uuid: "cfd414c3362447ba9037167ba3f86728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd414c3362447ba9037167ba3f86728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38991 } } }
I20260809 16:15:11.018054 17307 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:11.018362 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:11.012739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41203 (local address 127.0.0.1:38991)
0809 16:15:11.012924 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:11.012933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:11.012965 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:15:11.013480 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:11.013490 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:11.013555 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:11.013710 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:11.013870 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:11.015288 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:11.016110 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:11.016582 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:11.016682 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:11.017498 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:11.017502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:11.017503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:11.017717 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:11.017759 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:11.017775 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:11.018045 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:11.018092 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 16:15:11.018125 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260809 16:15:11.018558 17316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:11.019127 17316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:11.019611 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:11.021056 17316 catalog_manager.cc:1347] Generated new cluster ID: 0f3f6afc4a6c4bc1867d4eb49f1617ff
I20260809 16:15:11.021121 17316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:11.024045 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:11.021361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34107 (local address 127.0.0.1:38991)
0809 16:15:11.021396 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:11.021397 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:11.021427 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:15:11.021447 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:11.021448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:11.021456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:11.021497 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:11.021637 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:11.022117 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:11.022908 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:11.023139 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:11.023194 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:11.023809 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:11.023811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:11.023811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:11.023816 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:11.023830 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:11.023835 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:11.023884 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:11.023960 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:15:11.023970 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:15:11.031358 17316 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:11.031893 17316 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:11.035537 17316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfd414c3362447ba9037167ba3f86728: Generated new TSK 0
I20260809 16:15:11.035735 17316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:11.040764 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17179
W20260809 16:15:11.057401 17144 connection.cc:535] client connection to 127.0.0.1:38991 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38991): Connection reset by peer (error 104)
2026-08-09T16:15:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:11 dist-test-slave-fgb3 krb5kdc[17356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:11 dist-test-slave-fgb3 krb5kdc[17356](info): set up 2 sockets
Aug 09 16:15:11 dist-test-slave-fgb3 krb5kdc[17356](info): commencing operation
krb5kdc: starting...
W20260809 16:15:12.191691 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:12 dist-test-slave-fgb3 krb5kdc[17356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:12.311188 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:12.391959 17374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:12.392154 17374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:12.392176 17374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:12.393328 17374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:12.393369 17374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:12.393381 17374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:12.393404 17374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:12.393426 17374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:12.394913 17374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:12.395146 17374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:12.395357 17374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:12.399996 17380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:12.399991 17383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:12.400203 17374 server_base.cc:1034] running on GCE node
W20260809 16:15:12.400045 17381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:12.400619 17374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:12.400919 17374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:12.402088 17374 hybrid_clock.cc:648] HybridClock initialized: now 1786292112402062 us; error 51 us; skew 500 ppm
Aug 09 16:15:12 dist-test-slave-fgb3 krb5kdc[17356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:12.404595 17374 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:12.405047 17374 webserver.cc:460] Webserver started at http://127.0.0.1:42119/ using document root <none> and password file <none>
I20260809 16:15:12.405248 17374 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:12.405295 17374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:12.405421 17374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:12.406505 17374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "b3b7927dac504dd6a28b3ce3fd9515ac"
format_stamp: "Formatted at 2026-08-09 16:15:12 on dist-test-slave-fgb3"
server_key: "af1db20d59734092e0a6b40617915021"
server_key_iv: "cf1ffd8c9d639f06ab72d2b454e1875c"
server_key_version: "encryptionkey@0"
I20260809 16:15:12.406842 17374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "b3b7927dac504dd6a28b3ce3fd9515ac"
format_stamp: "Formatted at 2026-08-09 16:15:12 on dist-test-slave-fgb3"
server_key: "af1db20d59734092e0a6b40617915021"
server_key_iv: "cf1ffd8c9d639f06ab72d2b454e1875c"
server_key_version: "encryptionkey@0"
I20260809 16:15:12.408089 17374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:15:12.408717 17397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:12.408875 17374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:12.408970 17374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "b3b7927dac504dd6a28b3ce3fd9515ac"
format_stamp: "Formatted at 2026-08-09 16:15:12 on dist-test-slave-fgb3"
server_key: "af1db20d59734092e0a6b40617915021"
server_key_iv: "cf1ffd8c9d639f06ab72d2b454e1875c"
server_key_version: "encryptionkey@0"
I20260809 16:15:12.409029 17374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:12.412225 17374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:12.413829 17374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:12.413967 17374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:12.417819 17374 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36369
I20260809 16:15:12.417860 17478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36369 every 8 connection(s)
I20260809 16:15:12.418165 17374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:12.419152 17479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:12.421404 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac: Bootstrap starting.
I20260809 16:15:12.422012 17479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:12.422272 17479 log.cc:824] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:12.422921 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac: No bootstrap required, opened a new log
I20260809 16:15:12.424495 17479 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 16:15:12.424633 17479 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:12.424654 17479 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b7927dac504dd6a28b3ce3fd9515ac, State: Initialized, Role: FOLLOWER
I20260809 16:15:12.424757 17479 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 16:15:12.424811 17479 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:12.424831 17479 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:12.424860 17479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:12.425340 17479 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 16:15:12.425428 17479 leader_election.cc:302] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3b7927dac504dd6a28b3ce3fd9515ac; no voters:
I20260809 16:15:12.425546 17479 leader_election.cc:288] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:12.425594 17486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:12.425742 17486 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [term 1 LEADER]: Becoming Leader. State: Replica: b3b7927dac504dd6a28b3ce3fd9515ac, State: Running, Role: LEADER
I20260809 16:15:12.425796 17479 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:12.425858 17486 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 16:15:12.426069 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17374
I20260809 16:15:12.426192 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:12.426409 17282 external_mini_cluster.cc:1428] Setting key 8537982773596ab8ca8c9e2c3dbb7a0b
I20260809 16:15:12.426401 17488 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } } }
I20260809 16:15:12.426488 17488 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:12.426798 17491 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3b7927dac504dd6a28b3ce3fd9515ac. Latest consensus state: current_term: 1 leader_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b7927dac504dd6a28b3ce3fd9515ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } } }
I20260809 16:15:12.426868 17491 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:12.427265 17500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:12.427950 17500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:12.428591 17507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:12.426856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54209 (local address 127.0.0.1:36369)
0809 16:15:12.427106 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:12.427115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:12.427147 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:15:12.427306 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:12.427314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:12.427345 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:12.427488 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:12.428138 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:12.428142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:12.428142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:12.428147 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:12.428193 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:12.428208 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:12.428276 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:12.428363 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:15:12.428405 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
I20260809 16:15:12.429610 17500 catalog_manager.cc:1347] Generated new cluster ID: 84d1cdf09f8b40789515c59305f14cf3
I20260809 16:15:12.429657 17500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:12.429649 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:12.431926 17507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:12.430960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49993 (local address 127.0.0.1:36369)
0809 16:15:12.430995 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:12.430997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:12.431108 (+ 111us) server_negotiation.cc:407] Connection header received
0809 16:15:12.431161 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:12.431162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:12.431167 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:12.431211 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:12.431709 (+ 498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:12.431711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:12.431711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:12.431715 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:12.431730 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:12.431734 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:12.431780 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:12.431836 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 16:15:12.431849 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:15:12.463917 17500 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:12.464123 17500 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:12.467702 17500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3b7927dac504dd6a28b3ce3fd9515ac: Generated new TSK 0
I20260809 16:15:12.467885 17500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:12.474540 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17374
2026-08-09T16:15:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:12 dist-test-slave-fgb3 krb5kdc[17528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:12 dist-test-slave-fgb3 krb5kdc[17528](info): set up 2 sockets
Aug 09 16:15:12 dist-test-slave-fgb3 krb5kdc[17528](info): commencing operation
krb5kdc: starting...
W20260809 16:15:13.628459 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:13 dist-test-slave-fgb3 krb5kdc[17528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:13.744498 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:13.825821 17546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:13.826030 17546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:13.826061 17546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:13.827212 17546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:13.827270 17546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:13.827293 17546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:13.827314 17546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:13.827342 17546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:13.828806 17546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:13.829052 17546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:13.829278 17546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:13.834136 17554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:13.834127 17555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:13.834128 17558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:13.834180 17546 server_base.cc:1034] running on GCE node
I20260809 16:15:13.834527 17546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:13.834810 17546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:13.835969 17546 hybrid_clock.cc:648] HybridClock initialized: now 1786292113835948 us; error 37 us; skew 500 ppm
Aug 09 16:15:13 dist-test-slave-fgb3 krb5kdc[17528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292113, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:13.838239 17546 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:13.838670 17546 webserver.cc:460] Webserver started at http://127.0.0.1:34917/ using document root <none> and password file <none>
I20260809 16:15:13.838862 17546 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:13.838919 17546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:13.839043 17546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:13.840068 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "29e9c34b82a449b087a46d8d378148c6"
format_stamp: "Formatted at 2026-08-09 16:15:13 on dist-test-slave-fgb3"
server_key: "ad4aa58564e9138de6ed61919fe7f33c"
server_key_iv: "4a96864dd18374d4f0cc2f7b035a40f7"
server_key_version: "encryptionkey@0"
I20260809 16:15:13.840409 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "29e9c34b82a449b087a46d8d378148c6"
format_stamp: "Formatted at 2026-08-09 16:15:13 on dist-test-slave-fgb3"
server_key: "ad4aa58564e9138de6ed61919fe7f33c"
server_key_iv: "4a96864dd18374d4f0cc2f7b035a40f7"
server_key_version: "encryptionkey@0"
I20260809 16:15:13.841622 17546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:13.842263 17568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:13.842417 17546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:13.842504 17546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "29e9c34b82a449b087a46d8d378148c6"
format_stamp: "Formatted at 2026-08-09 16:15:13 on dist-test-slave-fgb3"
server_key: "ad4aa58564e9138de6ed61919fe7f33c"
server_key_iv: "4a96864dd18374d4f0cc2f7b035a40f7"
server_key_version: "encryptionkey@0"
I20260809 16:15:13.842566 17546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:13.853513 17546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:13.855240 17546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:13.855377 17546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:13.859352 17546 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36175
I20260809 16:15:13.859400 17646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36175 every 8 connection(s)
I20260809 16:15:13.859714 17546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:13.860558 17647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:13.862931 17647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6: Bootstrap starting.
I20260809 16:15:13.863502 17647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:13.863761 17647 log.cc:824] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:13.864413 17647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6: No bootstrap required, opened a new log
I20260809 16:15:13.865949 17647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e9c34b82a449b087a46d8d378148c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36175 } }
I20260809 16:15:13.866091 17647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:13.866111 17647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e9c34b82a449b087a46d8d378148c6, State: Initialized, Role: FOLLOWER
I20260809 16:15:13.866248 17647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e9c34b82a449b087a46d8d378148c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36175 } }
I20260809 16:15:13.866335 17647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:13.866376 17647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:13.866422 17647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:13.866928 17647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e9c34b82a449b087a46d8d378148c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36175 } }
I20260809 16:15:13.867027 17647 leader_election.cc:302] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29e9c34b82a449b087a46d8d378148c6; no voters:
I20260809 16:15:13.867177 17647 leader_election.cc:288] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:13.867218 17653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:13.867375 17653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [term 1 LEADER]: Becoming Leader. State: Replica: 29e9c34b82a449b087a46d8d378148c6, State: Running, Role: LEADER
I20260809 16:15:13.867422 17647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:13.867482 17653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e9c34b82a449b087a46d8d378148c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36175 } }
I20260809 16:15:13.867851 17655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e9c34b82a449b087a46d8d378148c6. Latest consensus state: current_term: 1 leader_uuid: "29e9c34b82a449b087a46d8d378148c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e9c34b82a449b087a46d8d378148c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36175 } } }
I20260809 16:15:13.867967 17655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:13.867982 17654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29e9c34b82a449b087a46d8d378148c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e9c34b82a449b087a46d8d378148c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36175 } } }
I20260809 16:15:13.868073 17654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:13.868487 17668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:13.869019 17668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:13.869100 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17546
I20260809 16:15:13.869221 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:13.869405 17282 external_mini_cluster.cc:1428] Setting key 87608faf4ec339a7ccc74bbbb5cdd916
I20260809 16:15:13.870703 17668 catalog_manager.cc:1347] Generated new cluster ID: c94c20df590d4bd5ae4d70215d3b4d7d
I20260809 16:15:13.870754 17668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:15:13 dist-test-slave-fgb3 krb5kdc[17528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:15:13.881039 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:13.869853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58877 (local address 127.0.0.1:36175)
0809 16:15:13.870364 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:13.870373 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:13.870406 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:15:13.870526 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:13.870534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:13.870591 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:13.870723 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:13.871616 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:13.873037 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:13.874100 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:13.874703 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:13.874794 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:13.877528 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:13.877533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:13.877533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:13.877546 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:13.879515 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:13.879867 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:13.879869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:13.879870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:13.879981 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:13.880152 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:13.880153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:13.880153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:13.880457 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:13.880603 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:13.880669 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:15:13.880800 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":67,"threads_started":1}
I20260809 16:15:13.882061 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:13.889979 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:13.883485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32883 (local address 127.0.0.1:36175)
0809 16:15:13.883518 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:13.883519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:13.883568 (+ 49us) server_negotiation.cc:407] Connection header received
0809 16:15:13.883634 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:13.883634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:13.883643 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:13.883696 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:13.884268 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:13.884962 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:13.885911 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:13.886202 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:13.886254 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:13.887510 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:13.887511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:13.887512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:13.887518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:13.888926 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:13.889263 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:13.889264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:13.889264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:13.889309 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:13.889475 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:13.889475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:13.889476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:13.889581 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:13.889656 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:13.889826 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 16:15:13.889894 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:13.898291 17668 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:13.899053 17668 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:13.905375 17668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29e9c34b82a449b087a46d8d378148c6: Generated new TSK 0
I20260809 16:15:13.905555 17668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:13.908829 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17546
2026-08-09T16:15:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:13 dist-test-slave-fgb3 krb5kdc[17700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:13 dist-test-slave-fgb3 krb5kdc[17700](info): set up 2 sockets
Aug 09 16:15:13 dist-test-slave-fgb3 krb5kdc[17700](info): commencing operation
krb5kdc: starting...
W20260809 16:15:15.058907 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:15 dist-test-slave-fgb3 krb5kdc[17700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:15.178970 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:15.260524 17718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:15.260743 17718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:15.260778 17718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:15.261911 17718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:15.261958 17718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:15.261981 17718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:15.262001 17718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:15.262027 17718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:15.263581 17718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:15.263832 17718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:15.264043 17718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:15.268657 17730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:15.268687 17728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:15.268759 17726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:15.268793 17718 server_base.cc:1034] running on GCE node
I20260809 16:15:15.269145 17718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:15.269429 17718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:15.270565 17718 hybrid_clock.cc:648] HybridClock initialized: now 1786292115270553 us; error 28 us; skew 500 ppm
Aug 09 16:15:15 dist-test-slave-fgb3 krb5kdc[17700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292115, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:15.276139 17718 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:15.276564 17718 webserver.cc:460] Webserver started at http://127.0.0.1:45167/ using document root <none> and password file <none>
I20260809 16:15:15.276738 17718 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:15.276779 17718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:15.276878 17718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:15.277860 17718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "822643f3c4164f38b576422391129b95"
format_stamp: "Formatted at 2026-08-09 16:15:15 on dist-test-slave-fgb3"
server_key: "7e62ee44456f6b6a944bf895f021a123"
server_key_iv: "ff8f360c37d2f348b3cbbf697a06a825"
server_key_version: "encryptionkey@0"
I20260809 16:15:15.278174 17718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "822643f3c4164f38b576422391129b95"
format_stamp: "Formatted at 2026-08-09 16:15:15 on dist-test-slave-fgb3"
server_key: "7e62ee44456f6b6a944bf895f021a123"
server_key_iv: "ff8f360c37d2f348b3cbbf697a06a825"
server_key_version: "encryptionkey@0"
I20260809 16:15:15.279479 17718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:15:15.280144 17743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:15.280326 17718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:15.280426 17718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "822643f3c4164f38b576422391129b95"
format_stamp: "Formatted at 2026-08-09 16:15:15 on dist-test-slave-fgb3"
server_key: "7e62ee44456f6b6a944bf895f021a123"
server_key_iv: "ff8f360c37d2f348b3cbbf697a06a825"
server_key_version: "encryptionkey@0"
I20260809 16:15:15.280486 17718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:15.289320 17718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:15.291272 17718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:15.291389 17718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:15.295393 17718 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35209
I20260809 16:15:15.295437 17832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35209 every 8 connection(s)
I20260809 16:15:15.295740 17718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:15.296569 17833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:15.298843 17833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95: Bootstrap starting.
I20260809 16:15:15.299393 17833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:15.299651 17833 log.cc:824] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:15.300300 17833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95: No bootstrap required, opened a new log
I20260809 16:15:15.301833 17833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822643f3c4164f38b576422391129b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35209 } }
I20260809 16:15:15.301967 17833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:15.301990 17833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 822643f3c4164f38b576422391129b95, State: Initialized, Role: FOLLOWER
I20260809 16:15:15.302129 17833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822643f3c4164f38b576422391129b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35209 } }
I20260809 16:15:15.302206 17833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:15.302240 17833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:15.302296 17833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:15.302804 17833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822643f3c4164f38b576422391129b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35209 } }
I20260809 16:15:15.302891 17833 leader_election.cc:302] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 822643f3c4164f38b576422391129b95; no voters:
I20260809 16:15:15.303035 17833 leader_election.cc:288] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:15.303076 17837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:15.303227 17837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [term 1 LEADER]: Becoming Leader. State: Replica: 822643f3c4164f38b576422391129b95, State: Running, Role: LEADER
I20260809 16:15:15.303295 17833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:15.303344 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17718
I20260809 16:15:15.303382 17837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822643f3c4164f38b576422391129b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35209 } }
I20260809 16:15:15.303464 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:15.303660 17282 external_mini_cluster.cc:1428] Setting key 5448c46e6f454140be61d2bfda0b8b09
I20260809 16:15:15.304013 17841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "822643f3c4164f38b576422391129b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822643f3c4164f38b576422391129b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35209 } } }
I20260809 16:15:15.304085 17841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:15.304464 17844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 822643f3c4164f38b576422391129b95. Latest consensus state: current_term: 1 leader_uuid: "822643f3c4164f38b576422391129b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822643f3c4164f38b576422391129b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35209 } } }
I20260809 16:15:15.304539 17844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95 [sys.catalog]: This master's current role is: LEADER
W20260809 16:15:15.305090 17869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:15:15.305184 17869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:15.305282 17856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:15.305500 17856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:15.306803 17856 catalog_manager.cc:1347] Generated new cluster ID: 05e4db7f944f4fdf9b033e4a03ad1b53
I20260809 16:15:15.306847 17856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:15.309979 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:15.304565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37433 (local address 127.0.0.1:35209)
0809 16:15:15.304820 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:15.304830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:15.304862 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:15:15.304981 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:15.304987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:15.305028 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:15.305148 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:15.305383 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:15.306806 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:15.307940 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:15.308536 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:15.308638 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:15.309306 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:15.309310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:15.309310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:15.309514 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:15.309553 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:15.309568 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:15.309637 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:15.309693 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 16:15:15.309756 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":58,"threads_started":1}
I20260809 16:15:15.310879 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:15.321599 17856 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:15.322232 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:15.317246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60309 (local address 127.0.0.1:35209)
0809 16:15:15.317284 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:15.317285 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:15.318876 (+ 1591us) server_negotiation.cc:407] Connection header received
0809 16:15:15.318902 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:15.318903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:15.318929 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:15.318987 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:15.319174 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:15.319924 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:15.320909 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:15.321228 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:15.321295 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:15.321942 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:15.321944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:15.321944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:15.321949 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:15.321975 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:15.321980 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:15.322034 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:15.322095 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 16:15:15.322136 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":19}
I20260809 16:15:15.322283 17856 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:15.327908 17856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 822643f3c4164f38b576422391129b95: Generated new TSK 0
I20260809 16:15:15.328078 17856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:15.332600 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17718
2026-08-09T16:15:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:15 dist-test-slave-fgb3 krb5kdc[17894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:15 dist-test-slave-fgb3 krb5kdc[17894](info): set up 2 sockets
Aug 09 16:15:15 dist-test-slave-fgb3 krb5kdc[17894](info): commencing operation
krb5kdc: starting...
W20260809 16:15:16.480139 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:16 dist-test-slave-fgb3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:16.595338 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:16.676831 17920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:16.677047 17920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:16.677078 17920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:16.678253 17920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:16.678303 17920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:16.678342 17920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:16.678359 17920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:16.678378 17920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:16.679916 17920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:16.680172 17920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:16.680395 17920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:16.685053 17928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:16.685024 17929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:16.685221 17920 server_base.cc:1034] running on GCE node
W20260809 16:15:16.685031 17932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:16.685580 17920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:16.685853 17920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:16.686988 17920 hybrid_clock.cc:648] HybridClock initialized: now 1786292116686970 us; error 33 us; skew 500 ppm
Aug 09 16:15:16 dist-test-slave-fgb3 krb5kdc[17894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:16.689512 17920 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:16.689940 17920 webserver.cc:460] Webserver started at http://127.0.0.1:42939/ using document root <none> and password file <none>
I20260809 16:15:16.690140 17920 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:16.690188 17920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:16.690428 17920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:16.691570 17920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "73b285b6c20a42eb95a8be6e47d132a8"
format_stamp: "Formatted at 2026-08-09 16:15:16 on dist-test-slave-fgb3"
server_key: "aa93c5fa802a9cff8e35b09cd5b6af72"
server_key_iv: "a17d6faad107ed2b901d556ac80a34b6"
server_key_version: "encryptionkey@0"
I20260809 16:15:16.691905 17920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "73b285b6c20a42eb95a8be6e47d132a8"
format_stamp: "Formatted at 2026-08-09 16:15:16 on dist-test-slave-fgb3"
server_key: "aa93c5fa802a9cff8e35b09cd5b6af72"
server_key_iv: "a17d6faad107ed2b901d556ac80a34b6"
server_key_version: "encryptionkey@0"
I20260809 16:15:16.693200 17920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:16.693874 17942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:16.694044 17920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:16.694136 17920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "73b285b6c20a42eb95a8be6e47d132a8"
format_stamp: "Formatted at 2026-08-09 16:15:16 on dist-test-slave-fgb3"
server_key: "aa93c5fa802a9cff8e35b09cd5b6af72"
server_key_iv: "a17d6faad107ed2b901d556ac80a34b6"
server_key_version: "encryptionkey@0"
I20260809 16:15:16.694193 17920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:16.697363 17920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:16.699440 17920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:16.699545 17920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:16.703298 17920 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35897
I20260809 16:15:16.703363 18032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35897 every 8 connection(s)
I20260809 16:15:16.703637 17920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:16.704573 18034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:16.706840 18034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8: Bootstrap starting.
I20260809 16:15:16.707376 18034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:16.707607 18034 log.cc:824] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:16.708235 18034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8: No bootstrap required, opened a new log
I20260809 16:15:16.709774 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 17920
I20260809 16:15:16.709895 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:16.709856 18034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b285b6c20a42eb95a8be6e47d132a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35897 } }
I20260809 16:15:16.710041 18034 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:16.710081 18034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b285b6c20a42eb95a8be6e47d132a8, State: Initialized, Role: FOLLOWER
I20260809 16:15:16.710103 17282 external_mini_cluster.cc:1428] Setting key 80b9efd0aa00b6d5a41f9ab6ff9c8558
I20260809 16:15:16.710223 18034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b285b6c20a42eb95a8be6e47d132a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35897 } }
I20260809 16:15:16.710302 18034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:16.710354 18034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:16.710394 18034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:16.711315 18034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b285b6c20a42eb95a8be6e47d132a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35897 } }
I20260809 16:15:16.711416 18034 leader_election.cc:302] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73b285b6c20a42eb95a8be6e47d132a8; no voters:
I20260809 16:15:16.711563 18034 leader_election.cc:288] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:16.711607 18045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:16.711723 18045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [term 1 LEADER]: Becoming Leader. State: Replica: 73b285b6c20a42eb95a8be6e47d132a8, State: Running, Role: LEADER
I20260809 16:15:16.711817 18034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:16.711856 18045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b285b6c20a42eb95a8be6e47d132a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35897 } }
I20260809 16:15:16.712203 18045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73b285b6c20a42eb95a8be6e47d132a8. Latest consensus state: current_term: 1 leader_uuid: "73b285b6c20a42eb95a8be6e47d132a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b285b6c20a42eb95a8be6e47d132a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35897 } } }
I20260809 16:15:16.712234 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:16.710611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45025 (local address 127.0.0.1:35897)
0809 16:15:16.710799 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:16.710808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:16.710835 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:15:16.710938 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:16.710943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:16.710965 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:16.711068 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:16.711701 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:16.711705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:16.711705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:16.711709 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:16.711755 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:16.711770 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:16.711823 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:16.712018 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 16:15:16.712058 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260809 16:15:16.712440 18045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:16.712354 18049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73b285b6c20a42eb95a8be6e47d132a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b285b6c20a42eb95a8be6e47d132a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35897 } } }
I20260809 16:15:16.712836 18049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:16.713047 18059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:16.713436 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:16.713591 18059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:16.718941 18059 catalog_manager.cc:1347] Generated new cluster ID: a20aeec4e4ad47088ebb592a593a1aba
I20260809 16:15:16.718994 18059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:16.719702 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:16.718590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41117 (local address 127.0.0.1:35897)
0809 16:15:16.718629 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:16.718631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:16.718783 (+ 152us) server_negotiation.cc:407] Connection header received
0809 16:15:16.718848 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:16.718849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:16.718854 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:16.718883 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:16.719502 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:16.719503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:16.719503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:16.719508 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:16.719522 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:16.719527 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:16.719580 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:16.719631 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:15:16.719642 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:15:16.740156 18059 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:16.740356 18059 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:16.759912 18059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73b285b6c20a42eb95a8be6e47d132a8: Generated new TSK 0
I20260809 16:15:16.760128 18059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:16.765856 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 17920
2026-08-09T16:15:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:16 dist-test-slave-fgb3 krb5kdc[18088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:16 dist-test-slave-fgb3 krb5kdc[18088](info): set up 2 sockets
Aug 09 16:15:16 dist-test-slave-fgb3 krb5kdc[18088](info): commencing operation
krb5kdc: starting...
W20260809 16:15:17.908622 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:17 dist-test-slave-fgb3 krb5kdc[18088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:18.023151 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:18.103739 18106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:18.103932 18106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:18.103962 18106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:18.105048 18106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:18.105089 18106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:18.105108 18106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:18.105121 18106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:18.105137 18106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:18.106642 18106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:18.106889 18106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:18.107092 18106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:18.111517 18113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:18.111531 18116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:18.111789 18106 server_base.cc:1034] running on GCE node
W20260809 16:15:18.111537 18119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:18.112053 18106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:18.112365 18106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:18.113515 18106 hybrid_clock.cc:648] HybridClock initialized: now 1786292118113498 us; error 28 us; skew 500 ppm
Aug 09 16:15:18 dist-test-slave-fgb3 krb5kdc[18088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292118, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:18.115850 18106 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:18.116261 18106 webserver.cc:460] Webserver started at http://127.0.0.1:42501/ using document root <none> and password file <none>
I20260809 16:15:18.116479 18106 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:18.116525 18106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:18.116643 18106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:18.117611 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "e91409d354c349ad8a7cf7d77bc894c6"
format_stamp: "Formatted at 2026-08-09 16:15:18 on dist-test-slave-fgb3"
server_key: "aee3e32cd94898c8d59dabced9ba18f0"
server_key_iv: "a0ab2a6a1096804d3387ede78104d034"
server_key_version: "encryptionkey@0"
I20260809 16:15:18.117929 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "e91409d354c349ad8a7cf7d77bc894c6"
format_stamp: "Formatted at 2026-08-09 16:15:18 on dist-test-slave-fgb3"
server_key: "aee3e32cd94898c8d59dabced9ba18f0"
server_key_iv: "a0ab2a6a1096804d3387ede78104d034"
server_key_version: "encryptionkey@0"
I20260809 16:15:18.119199 18106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:18.119904 18129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:18.120064 18106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:18.120147 18106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "e91409d354c349ad8a7cf7d77bc894c6"
format_stamp: "Formatted at 2026-08-09 16:15:18 on dist-test-slave-fgb3"
server_key: "aee3e32cd94898c8d59dabced9ba18f0"
server_key_iv: "a0ab2a6a1096804d3387ede78104d034"
server_key_version: "encryptionkey@0"
I20260809 16:15:18.120205 18106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:18.138192 18106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:18.140143 18106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:18.140270 18106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:18.144066 18106 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46663
I20260809 16:15:18.144100 18220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46663 every 8 connection(s)
I20260809 16:15:18.144423 18106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:18.145211 18222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:18.147423 18222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6: Bootstrap starting.
I20260809 16:15:18.147773 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18106
I20260809 16:15:18.147902 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:18.148069 18222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:18.148094 17282 external_mini_cluster.cc:1428] Setting key 84c9c906f362b2e2ffb781e4f39032da
I20260809 16:15:18.148375 18222 log.cc:824] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:18.149130 18222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6: No bootstrap required, opened a new log
I20260809 16:15:18.151436 18222 raft_consensus.cc:348] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91409d354c349ad8a7cf7d77bc894c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 16:15:18.151598 18222 raft_consensus.cc:374] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:18.151631 18222 raft_consensus.cc:729] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91409d354c349ad8a7cf7d77bc894c6, State: Initialized, Role: FOLLOWER
I20260809 16:15:18.151753 18222 consensus_queue.cc:260] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91409d354c349ad8a7cf7d77bc894c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 16:15:18.151827 18222 raft_consensus.cc:388] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:18.151849 18222 raft_consensus.cc:482] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:18.151899 18222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:18.152400 18222 raft_consensus.cc:504] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91409d354c349ad8a7cf7d77bc894c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 16:15:18.152506 18222 leader_election.cc:302] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e91409d354c349ad8a7cf7d77bc894c6; no voters:
I20260809 16:15:18.152654 18222 leader_election.cc:288] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:18.152700 18231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:18.152884 18222 sys_catalog.cc:564] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:18.152890 18231 raft_consensus.cc:686] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [term 1 LEADER]: Becoming Leader. State: Replica: e91409d354c349ad8a7cf7d77bc894c6, State: Running, Role: LEADER
I20260809 16:15:18.153096 18231 consensus_queue.cc:237] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91409d354c349ad8a7cf7d77bc894c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 16:15:18.153545 18233 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e91409d354c349ad8a7cf7d77bc894c6. Latest consensus state: current_term: 1 leader_uuid: "e91409d354c349ad8a7cf7d77bc894c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91409d354c349ad8a7cf7d77bc894c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } } }
I20260809 16:15:18.153623 18233 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:18.153857 18247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:18.154297 18247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:18.154480 18232 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e91409d354c349ad8a7cf7d77bc894c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91409d354c349ad8a7cf7d77bc894c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } } }
I20260809 16:15:18.154688 18232 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6 [sys.catalog]: This master's current role is: LEADER
Aug 09 16:15:18 dist-test-slave-fgb3 krb5kdc[18088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:15:18.155972 18247 catalog_manager.cc:1347] Generated new cluster ID: f4cb5632ab304c7e81e48017c50ad62d
I20260809 16:15:18.156028 18247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:18.159041 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:18.148620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:58751 (local address 127.0.0.1:46663)
0809 16:15:18.148784 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:18.148804 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:18.148837 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:15:18.149390 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:18.149399 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:18.149463 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:18.149613 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:18.150209 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:18.151621 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:18.152443 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:18.153045 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:18.153164 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:18.155919 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:18.155923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:18.155923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:18.155935 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:18.157413 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:18.157775 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:18.157778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:18.157779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:18.157862 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:18.158081 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:18.158082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:18.158083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:18.158348 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:18.158516 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:18.158774 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 16:15:18.158885 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260809 16:15:18.160063 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:18.166821 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:18.161338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:53511 (local address 127.0.0.1:46663)
0809 16:15:18.161378 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:18.161380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:18.161452 (+ 72us) server_negotiation.cc:407] Connection header received
0809 16:15:18.161493 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:18.161494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:18.161504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:18.161547 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:18.162019 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:18.162568 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:18.163230 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:18.163452 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:18.163505 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:18.164536 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:18.164537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:18.164538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:18.164545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:18.165737 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:18.166072 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:18.166073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:18.166074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:18.166124 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:18.166337 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:18.166338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:18.166338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:18.166459 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:18.166550 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:18.166684 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 16:15:18.166740 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:15:18.172730 18247 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:18.173264 18247 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:18.181667 18247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e91409d354c349ad8a7cf7d77bc894c6: Generated new TSK 0
I20260809 16:15:18.181833 18247 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:18.184651 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18106
W20260809 16:15:18.197813 18083 connection.cc:535] client connection to 127.0.0.1:46663 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46663): Connection reset by peer (error 104)
2026-08-09T16:15:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:18 dist-test-slave-fgb3 krb5kdc[18271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:18 dist-test-slave-fgb3 krb5kdc[18271](info): set up 2 sockets
Aug 09 16:15:18 dist-test-slave-fgb3 krb5kdc[18271](info): commencing operation
krb5kdc: starting...
W20260809 16:15:19.339416 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:19 dist-test-slave-fgb3 krb5kdc[18271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:19.459489 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:19.539562 18296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:19.539752 18296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:19.539773 18296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:19.540886 18296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:19.540927 18296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:19.540940 18296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:19.540952 18296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:19.540967 18296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:19.542366 18296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:19.542553 18296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:19.542713 18296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:19.547088 18309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:19.547114 18311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:19.547130 18308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:19.547258 18296 server_base.cc:1034] running on GCE node
I20260809 16:15:19.547585 18296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:19.547845 18296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:19.548979 18296 hybrid_clock.cc:648] HybridClock initialized: now 1786292119548966 us; error 30 us; skew 500 ppm
Aug 09 16:15:19 dist-test-slave-fgb3 krb5kdc[18271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292119, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:19.551301 18296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:19.551728 18296 webserver.cc:460] Webserver started at http://127.0.0.1:40401/ using document root <none> and password file <none>
I20260809 16:15:19.551916 18296 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:19.551962 18296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:19.552086 18296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:19.553066 18296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "b0670782767148aca71187f21c6e007e"
format_stamp: "Formatted at 2026-08-09 16:15:19 on dist-test-slave-fgb3"
server_key: "47fe18d0aa3858ad36900104bc9ec8da"
server_key_iv: "d1cae2ebed2f9b428c77478a74969d48"
server_key_version: "encryptionkey@0"
I20260809 16:15:19.553404 18296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "b0670782767148aca71187f21c6e007e"
format_stamp: "Formatted at 2026-08-09 16:15:19 on dist-test-slave-fgb3"
server_key: "47fe18d0aa3858ad36900104bc9ec8da"
server_key_iv: "d1cae2ebed2f9b428c77478a74969d48"
server_key_version: "encryptionkey@0"
I20260809 16:15:19.554634 18296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:19.555343 18320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:19.555502 18296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:19.555588 18296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "b0670782767148aca71187f21c6e007e"
format_stamp: "Formatted at 2026-08-09 16:15:19 on dist-test-slave-fgb3"
server_key: "47fe18d0aa3858ad36900104bc9ec8da"
server_key_iv: "d1cae2ebed2f9b428c77478a74969d48"
server_key_version: "encryptionkey@0"
I20260809 16:15:19.555644 18296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:19.566468 18296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:19.568071 18296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:19.568202 18296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:19.572067 18296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46797
I20260809 16:15:19.572103 18406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46797 every 8 connection(s)
I20260809 16:15:19.572423 18296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:19.573298 18407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:19.574223 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18296
I20260809 16:15:19.574358 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:19.574538 17282 external_mini_cluster.cc:1428] Setting key 6dd432fa801272871cba2b2e96b4e2f0
I20260809 16:15:19.576028 18407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e: Bootstrap starting.
I20260809 16:15:19.576639 18407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:19.576915 18407 log.cc:824] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:19.577689 18407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e: No bootstrap required, opened a new log
I20260809 16:15:19.579919 18407 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0670782767148aca71187f21c6e007e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260809 16:15:19.580073 18407 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:19.580097 18407 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0670782767148aca71187f21c6e007e, State: Initialized, Role: FOLLOWER
I20260809 16:15:19.580165 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:19.575036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:58743 (local address 127.0.0.1:46797)
0809 16:15:19.575197 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:19.575206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:19.575230 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:15:19.575732 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:19.575739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:19.575787 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:19.575937 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:19.576192 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:19.577214 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:19.578342 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:19.578781 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:19.578860 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:19.579557 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:19.579561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:19.579561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:19.579712 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:19.579741 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:19.579751 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:19.579814 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:19.579924 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 16:15:19.579951 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260809 16:15:19.580219 18407 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0670782767148aca71187f21c6e007e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260809 16:15:19.580296 18407 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:19.580332 18407 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:19.580377 18407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:19.581028 18407 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0670782767148aca71187f21c6e007e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260809 16:15:19.581113 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:19.581139 18407 leader_election.cc:302] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0670782767148aca71187f21c6e007e; no voters:
I20260809 16:15:19.581297 18407 leader_election.cc:288] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:19.581329 18417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:19.581496 18417 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [term 1 LEADER]: Becoming Leader. State: Replica: b0670782767148aca71187f21c6e007e, State: Running, Role: LEADER
I20260809 16:15:19.581516 18407 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:19.581651 18417 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0670782767148aca71187f21c6e007e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } }
I20260809 16:15:19.582060 18419 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0670782767148aca71187f21c6e007e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0670782767148aca71187f21c6e007e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } } }
I20260809 16:15:19.582083 18418 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0670782767148aca71187f21c6e007e. Latest consensus state: current_term: 1 leader_uuid: "b0670782767148aca71187f21c6e007e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0670782767148aca71187f21c6e007e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46797 } } }
I20260809 16:15:19.582144 18419 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:19.582159 18418 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:19.583087 18429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:19.583709 18429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:19.585048 18429 catalog_manager.cc:1347] Generated new cluster ID: 86eb9e4abbbd49548c4b11062ee944fb
I20260809 16:15:19.585103 18429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:19.586071 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:19.582604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:42973 (local address 127.0.0.1:46797)
0809 16:15:19.582644 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:19.582645 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:19.582790 (+ 145us) server_negotiation.cc:407] Connection header received
0809 16:15:19.582884 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:19.582884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:19.582895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:19.582943 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:19.583143 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:19.583788 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:19.584774 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:19.585064 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:19.585132 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:19.585831 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:19.585832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:19.585833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:19.585837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:19.585849 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:19.585853 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:19.585888 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:19.585996 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:15:19.586005 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:15:19.600392 18429 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:19.600891 18429 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:19.603780 18429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0670782767148aca71187f21c6e007e: Generated new TSK 0
I20260809 16:15:19.603935 18429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:19.611016 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18296
W20260809 16:15:19.637629 18268 connection.cc:535] client connection to 127.0.0.1:46797 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46797): Connection reset by peer (error 104)
2026-08-09T16:15:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:19 dist-test-slave-fgb3 krb5kdc[18454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:19 dist-test-slave-fgb3 krb5kdc[18454](info): set up 2 sockets
Aug 09 16:15:19 dist-test-slave-fgb3 krb5kdc[18454](info): commencing operation
krb5kdc: starting...
W20260809 16:15:20.766485 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:20 dist-test-slave-fgb3 krb5kdc[18454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:20.888584 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:20.972398 18475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:20.972606 18475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:20.972627 18475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:20.973768 18475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:20.973810 18475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:20.973822 18475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:20.973845 18475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:20.973867 18475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:20.975347 18475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:20.975586 18475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:20.975792 18475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:20.980331 18485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:20.980417 18480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:20.980485 18482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:20.980551 18475 server_base.cc:1034] running on GCE node
I20260809 16:15:20.980846 18475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:20.981155 18475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:20.982307 18475 hybrid_clock.cc:648] HybridClock initialized: now 1786292120982288 us; error 28 us; skew 500 ppm
Aug 09 16:15:20 dist-test-slave-fgb3 krb5kdc[18454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292120, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:20.989959 18475 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:20.992453 18475 webserver.cc:460] Webserver started at http://127.0.0.1:38989/ using document root <none> and password file <none>
I20260809 16:15:20.992704 18475 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:20.992754 18475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:20.992887 18475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:20.993979 18475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "c60e9d9030b74046870250b33ef0030b"
format_stamp: "Formatted at 2026-08-09 16:15:20 on dist-test-slave-fgb3"
server_key: "b68592860ce1ef0b882d34b065da5c14"
server_key_iv: "56a31db018a80273a13a1e94db6ecf59"
server_key_version: "encryptionkey@0"
I20260809 16:15:20.994339 18475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "c60e9d9030b74046870250b33ef0030b"
format_stamp: "Formatted at 2026-08-09 16:15:20 on dist-test-slave-fgb3"
server_key: "b68592860ce1ef0b882d34b065da5c14"
server_key_iv: "56a31db018a80273a13a1e94db6ecf59"
server_key_version: "encryptionkey@0"
I20260809 16:15:20.995633 18475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:15:20.996301 18498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:20.996456 18475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:20.996544 18475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "c60e9d9030b74046870250b33ef0030b"
format_stamp: "Formatted at 2026-08-09 16:15:20 on dist-test-slave-fgb3"
server_key: "b68592860ce1ef0b882d34b065da5c14"
server_key_iv: "56a31db018a80273a13a1e94db6ecf59"
server_key_version: "encryptionkey@0"
I20260809 16:15:20.996606 18475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:20.999949 18475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:21.001740 18475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:21.001880 18475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:21.005712 18475 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37533
I20260809 16:15:21.005752 18583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37533 every 8 connection(s)
I20260809 16:15:21.006057 18475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:21.007064 18584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:21.009303 18584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b: Bootstrap starting.
I20260809 16:15:21.009848 18584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:21.010108 18584 log.cc:824] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:21.010812 18584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b: No bootstrap required, opened a new log
I20260809 16:15:21.012353 18584 raft_consensus.cc:348] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60e9d9030b74046870250b33ef0030b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 16:15:21.012487 18584 raft_consensus.cc:374] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:21.012521 18584 raft_consensus.cc:729] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60e9d9030b74046870250b33ef0030b, State: Initialized, Role: FOLLOWER
I20260809 16:15:21.012661 18584 consensus_queue.cc:260] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60e9d9030b74046870250b33ef0030b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 16:15:21.012737 18584 raft_consensus.cc:388] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:21.012773 18584 raft_consensus.cc:482] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:21.012820 18584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:21.013331 18584 raft_consensus.cc:504] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60e9d9030b74046870250b33ef0030b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 16:15:21.013422 18584 leader_election.cc:302] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c60e9d9030b74046870250b33ef0030b; no voters:
I20260809 16:15:21.013571 18584 leader_election.cc:288] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:21.013607 18591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:21.013796 18591 raft_consensus.cc:686] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [term 1 LEADER]: Becoming Leader. State: Replica: c60e9d9030b74046870250b33ef0030b, State: Running, Role: LEADER
I20260809 16:15:21.013821 18584 sys_catalog.cc:564] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:21.013813 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18475
I20260809 16:15:21.013897 18591 consensus_queue.cc:237] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60e9d9030b74046870250b33ef0030b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 16:15:21.013934 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:21.014118 17282 external_mini_cluster.cc:1428] Setting key 9cafb8ac26cbc521a2071e9a4ff0763e
I20260809 16:15:21.014559 18593 sys_catalog.cc:455] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c60e9d9030b74046870250b33ef0030b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60e9d9030b74046870250b33ef0030b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } } }
I20260809 16:15:21.014647 18593 sys_catalog.cc:458] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:21.014966 18595 sys_catalog.cc:455] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c60e9d9030b74046870250b33ef0030b. Latest consensus state: current_term: 1 leader_uuid: "c60e9d9030b74046870250b33ef0030b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c60e9d9030b74046870250b33ef0030b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } } }
I20260809 16:15:21.015062 18595 sys_catalog.cc:458] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:21.015280 18607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:21.015810 18607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:21.016872 18610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:21.014783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:33885 (local address 127.0.0.1:37533)
0809 16:15:21.015070 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:21.015080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:21.015116 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:15:21.015240 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:21.015248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:21.015279 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:21.015682 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:21.016471 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:21.016475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:21.016475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:21.016479 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:21.016516 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:21.016526 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:21.016577 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:21.016658 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 16:15:21.016700 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":81,"threads_started":1}
I20260809 16:15:21.017424 18607 catalog_manager.cc:1347] Generated new cluster ID: 9c770f1bd40f444f97db86205b40c737
I20260809 16:15:21.017478 18607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:21.017765 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:21.020514 18610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:21.019330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:51629 (local address 127.0.0.1:37533)
0809 16:15:21.019360 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:21.019361 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:21.019610 (+ 249us) server_negotiation.cc:407] Connection header received
0809 16:15:21.019678 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:21.019679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:21.019683 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:21.019734 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:21.020302 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:21.020303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:21.020303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:21.020307 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:21.020320 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:21.020324 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:21.020369 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:21.020431 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 16:15:21.020443 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 16:15:21.029336 18607 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:21.029522 18607 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:21.040964 18607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c60e9d9030b74046870250b33ef0030b: Generated new TSK 0
I20260809 16:15:21.041704 18607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:21.047081 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18475
2026-08-09T16:15:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:21 dist-test-slave-fgb3 krb5kdc[18640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:21 dist-test-slave-fgb3 krb5kdc[18640](info): set up 2 sockets
Aug 09 16:15:21 dist-test-slave-fgb3 krb5kdc[18640](info): commencing operation
krb5kdc: starting...
W20260809 16:15:22.201186 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:22 dist-test-slave-fgb3 krb5kdc[18640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:22.319126 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:22.401518 18663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:22.401715 18663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:22.401736 18663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:22.402886 18663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:22.402927 18663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:22.402941 18663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:22.402951 18663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:22.402962 18663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:22.404425 18663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:22.404655 18663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:22.404867 18663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:15:22.409633 18663 server_base.cc:1034] running on GCE node
W20260809 16:15:22.409580 18671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:22.409571 18677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:22.409578 18674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:22.410012 18663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:22.410264 18663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:22.411417 18663 hybrid_clock.cc:648] HybridClock initialized: now 1786292122411398 us; error 32 us; skew 500 ppm
Aug 09 16:15:22 dist-test-slave-fgb3 krb5kdc[18640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292122, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:22.413826 18663 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:22.414271 18663 webserver.cc:460] Webserver started at http://127.0.0.1:44305/ using document root <none> and password file <none>
I20260809 16:15:22.414506 18663 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:22.414557 18663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:22.414700 18663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:22.415776 18663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f"
format_stamp: "Formatted at 2026-08-09 16:15:22 on dist-test-slave-fgb3"
server_key: "db7488ee7c8f1ece63a9355de1bfa636"
server_key_iv: "800e317fa0e658cb2e775cfb465fabf2"
server_key_version: "encryptionkey@0"
I20260809 16:15:22.416121 18663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f"
format_stamp: "Formatted at 2026-08-09 16:15:22 on dist-test-slave-fgb3"
server_key: "db7488ee7c8f1ece63a9355de1bfa636"
server_key_iv: "800e317fa0e658cb2e775cfb465fabf2"
server_key_version: "encryptionkey@0"
I20260809 16:15:22.417435 18663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:22.418159 18685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:22.418385 18663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:15:22.418479 18663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f"
format_stamp: "Formatted at 2026-08-09 16:15:22 on dist-test-slave-fgb3"
server_key: "db7488ee7c8f1ece63a9355de1bfa636"
server_key_iv: "800e317fa0e658cb2e775cfb465fabf2"
server_key_version: "encryptionkey@0"
I20260809 16:15:22.418550 18663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:22.429396 18663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:22.431427 18663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:22.431565 18663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:22.436117 18663 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33713
I20260809 16:15:22.436142 18771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33713 every 8 connection(s)
I20260809 16:15:22.436638 18663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:22.437599 18772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:22.439997 18772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f: Bootstrap starting.
I20260809 16:15:22.440583 18772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:22.440852 18772 log.cc:824] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:22.441583 18772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f: No bootstrap required, opened a new log
I20260809 16:15:22.443222 18772 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260809 16:15:22.443375 18772 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:22.443410 18772 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3f7df0cca534bceaa77fc4dbf83ca7f, State: Initialized, Role: FOLLOWER
I20260809 16:15:22.443552 18772 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260809 16:15:22.443610 18772 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:22.443642 18772 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:22.443686 18772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:22.444196 18772 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260809 16:15:22.444293 18772 leader_election.cc:302] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3f7df0cca534bceaa77fc4dbf83ca7f; no voters:
I20260809 16:15:22.444480 18772 leader_election.cc:288] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:22.444507 18776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:22.444614 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18663
I20260809 16:15:22.444677 18776 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [term 1 LEADER]: Becoming Leader. State: Replica: b3f7df0cca534bceaa77fc4dbf83ca7f, State: Running, Role: LEADER
I20260809 16:15:22.444738 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:22.444753 18772 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:22.444794 18776 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260809 16:15:22.444988 17282 external_mini_cluster.cc:1428] Setting key f15ea2c456a534e449831f77cb958c1c
I20260809 16:15:22.445089 18776 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } } }
I20260809 16:15:22.445217 18776 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:22.445353 18778 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3f7df0cca534bceaa77fc4dbf83ca7f. Latest consensus state: current_term: 1 leader_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3f7df0cca534bceaa77fc4dbf83ca7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } } }
I20260809 16:15:22.445493 18778 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:22.445508 18787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:22.446249 18787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:22.447897 18787 catalog_manager.cc:1347] Generated new cluster ID: 89c3fe8ecd7b43fa9eb2e8ee001f23ee
I20260809 16:15:22.447947 18787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:15:22 dist-test-slave-fgb3 krb5kdc[18640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:15:22.458853 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:22.445688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:46463 (local address 127.0.0.1:33713)
0809 16:15:22.445856 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:22.445866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:22.447590 (+ 1724us) server_negotiation.cc:407] Connection header received
0809 16:15:22.447741 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:22.447749 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:22.447812 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:22.447960 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:22.448892 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:22.450445 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:22.451525 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:22.452136 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:22.452231 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:22.455129 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:22.455133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:22.455134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:22.455146 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:22.457075 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:22.457490 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:22.457493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:22.457494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:22.457624 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:22.457828 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:22.457828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:22.457829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:22.458159 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:22.458347 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:22.458483 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 16:15:22.458588 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260809 16:15:22.459198 18787 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:22.460011 18787 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:22.460129 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:22.468235 18787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3f7df0cca534bceaa77fc4dbf83ca7f: Generated new TSK 0
I20260809 16:15:22.468549 18787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:22.469009 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:22.461673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:44067 (local address 127.0.0.1:33713)
0809 16:15:22.461715 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:22.461717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:22.461886 (+ 169us) server_negotiation.cc:407] Connection header received
0809 16:15:22.461943 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:22.461944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:22.461955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:22.461998 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:22.462666 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:22.463435 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:22.464492 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:22.464803 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:22.464855 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:22.466260 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:22.466261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:22.466262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:22.466268 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:22.467810 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:22.468198 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:22.468199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:22.468199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:22.468261 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:22.468528 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:22.468529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:22.468529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:22.468656 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:22.468763 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:22.468894 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:15:22.468951 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:15:22.470824 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18663
W20260809 16:15:22.484774 18638 connection.cc:535] client connection to 127.0.0.1:33713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33713): Connection reset by peer (error 104)
2026-08-09T16:15:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:22 dist-test-slave-fgb3 krb5kdc[18820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:22 dist-test-slave-fgb3 krb5kdc[18820](info): set up 2 sockets
Aug 09 16:15:22 dist-test-slave-fgb3 krb5kdc[18820](info): commencing operation
krb5kdc: starting...
W20260809 16:15:23.634027 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:23 dist-test-slave-fgb3 krb5kdc[18820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:23.751152 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39841
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:23.832057 18838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:23.832248 18838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:23.832289 18838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:23.833392 18838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:23.833434 18838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:23.833451 18838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:23.833467 18838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:23.833482 18838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:23.834964 18838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:23.835199 18838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:23.835403 18838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:23.840052 18847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:23.840117 18838 server_base.cc:1034] running on GCE node
W20260809 16:15:23.840055 18851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:23.840117 18846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:23.840523 18838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:23.840786 18838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:23.841928 18838 hybrid_clock.cc:648] HybridClock initialized: now 1786292123841911 us; error 32 us; skew 500 ppm
Aug 09 16:15:23 dist-test-slave-fgb3 krb5kdc[18820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292123, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:23.844419 18838 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:23.844847 18838 webserver.cc:460] Webserver started at http://127.0.0.1:44185/ using document root <none> and password file <none>
I20260809 16:15:23.845077 18838 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:23.845129 18838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:23.845269 18838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:23.846374 18838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472"
format_stamp: "Formatted at 2026-08-09 16:15:23 on dist-test-slave-fgb3"
server_key: "15664244d9b510dcb425c264863b5083"
server_key_iv: "217cd24855da414faa756d5f8119c308"
server_key_version: "encryptionkey@0"
I20260809 16:15:23.846757 18838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472"
format_stamp: "Formatted at 2026-08-09 16:15:23 on dist-test-slave-fgb3"
server_key: "15664244d9b510dcb425c264863b5083"
server_key_iv: "217cd24855da414faa756d5f8119c308"
server_key_version: "encryptionkey@0"
I20260809 16:15:23.848080 18838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:15:23.848855 18863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:23.849025 18838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:23.849117 18838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472"
format_stamp: "Formatted at 2026-08-09 16:15:23 on dist-test-slave-fgb3"
server_key: "15664244d9b510dcb425c264863b5083"
server_key_iv: "217cd24855da414faa756d5f8119c308"
server_key_version: "encryptionkey@0"
I20260809 16:15:23.849181 18838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:23.873530 18838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:23.875639 18838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:23.875815 18838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:23.880008 18838 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39841
I20260809 16:15:23.880059 18941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39841 every 8 connection(s)
I20260809 16:15:23.880422 18838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:23.881316 18944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:23.883661 18944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472: Bootstrap starting.
I20260809 16:15:23.884213 18944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:23.884495 18944 log.cc:824] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:23.885140 18944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472: No bootstrap required, opened a new log
I20260809 16:15:23.886003 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 18838
I20260809 16:15:23.886119 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:23.886353 17282 external_mini_cluster.cc:1428] Setting key 3f4c686ef39f3af69e0fe84eac117aa9
I20260809 16:15:23.887140 18944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260809 16:15:23.887295 18944 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:23.887316 18944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0d2ea489dd4cb1a2b377d6ba0a9472, State: Initialized, Role: FOLLOWER
I20260809 16:15:23.887427 18944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260809 16:15:23.887506 18944 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:23.887569 18944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:23.887624 18944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:23.888150 18944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260809 16:15:23.888239 18944 leader_election.cc:302] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e0d2ea489dd4cb1a2b377d6ba0a9472; no voters:
I20260809 16:15:23.888404 18944 leader_election.cc:288] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:23.888434 18953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:23.888577 18953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [term 1 LEADER]: Becoming Leader. State: Replica: 7e0d2ea489dd4cb1a2b377d6ba0a9472, State: Running, Role: LEADER
I20260809 16:15:23.888665 18944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:23.888707 18953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260809 16:15:23.889120 18959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e0d2ea489dd4cb1a2b377d6ba0a9472. Latest consensus state: current_term: 1 leader_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } } }
I20260809 16:15:23.889204 18959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:23.889706 18966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:23.889400 18958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0d2ea489dd4cb1a2b377d6ba0a9472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } } }
I20260809 16:15:23.890102 18958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:23.890225 18966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:23.891749 18966 catalog_manager.cc:1347] Generated new cluster ID: 2dc10d6997b34871b1eb408861883537
I20260809 16:15:23.891813 18966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:23.892177 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:23.886857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:54559 (local address 127.0.0.1:39841)
0809 16:15:23.887034 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:23.887041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:23.887065 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:15:23.887166 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:23.887173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:23.887229 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:23.887384 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:23.887759 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:23.888861 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:23.889953 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:23.890561 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:23.890668 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:23.891476 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:23.891479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:23.891479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:23.891638 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:23.891676 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:23.891692 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:23.891768 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:23.891952 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 16:15:23.891978 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260809 16:15:23.893113 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:23.897774 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:23.894637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:58075 (local address 127.0.0.1:39841)
0809 16:15:23.894670 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:23.894672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:23.894679 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:23.894698 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:23.894698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:23.894708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:23.894767 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:23.894884 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:23.895609 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:23.896536 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:23.896837 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:23.896909 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:23.897545 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:23.897546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:23.897546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:23.897551 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:23.897566 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:23.897571 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:23.897630 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:23.897685 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:15:23.897698 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:15:23.904150 18966 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:23.904613 18966 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:23.909580 18966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e0d2ea489dd4cb1a2b377d6ba0a9472: Generated new TSK 0
I20260809 16:15:23.909726 18966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:23.916985 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 18838
W20260809 16:15:23.929767 18817 connection.cc:535] client connection to 127.0.0.1:39841 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39841): Connection reset by peer (error 104)
2026-08-09T16:15:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:23 dist-test-slave-fgb3 krb5kdc[18996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:23 dist-test-slave-fgb3 krb5kdc[18996](info): set up 2 sockets
Aug 09 16:15:23 dist-test-slave-fgb3 krb5kdc[18996](info): commencing operation
krb5kdc: starting...
W20260809 16:15:25.066042 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:25 dist-test-slave-fgb3 krb5kdc[18996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:15:25.183199 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:25.264700 19013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:25.264909 19013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:25.264938 19013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:25.266050 19013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:25.266093 19013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:25.266111 19013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:25.266127 19013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:25.266141 19013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:25.267597 19013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40859
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:25.267844 19013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:25.268049 19013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:25.272581 19019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:25.272586 19023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:25.272596 19020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:25.272746 19013 server_base.cc:1034] running on GCE node
I20260809 16:15:25.273066 19013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:25.273334 19013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:25.274461 19013 hybrid_clock.cc:648] HybridClock initialized: now 1786292125274449 us; error 29 us; skew 500 ppm
Aug 09 16:15:25 dist-test-slave-fgb3 krb5kdc[18996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292125, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:25.276741 19013 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:15:25.277155 19013 webserver.cc:460] Webserver started at http://127.0.0.1:43851/ using document root <none> and password file <none>
I20260809 16:15:25.277354 19013 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:25.277400 19013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:25.277524 19013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:25.278606 19013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "7f48f391ba134f39a5863a6edd1ba26e"
format_stamp: "Formatted at 2026-08-09 16:15:25 on dist-test-slave-fgb3"
server_key: "0afe0e40c89ddf1221f5cb408a180d40"
server_key_iv: "d7a83605da93230e88483515c80e49e6"
server_key_version: "encryptionkey@0"
I20260809 16:15:25.278939 19013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "7f48f391ba134f39a5863a6edd1ba26e"
format_stamp: "Formatted at 2026-08-09 16:15:25 on dist-test-slave-fgb3"
server_key: "0afe0e40c89ddf1221f5cb408a180d40"
server_key_iv: "d7a83605da93230e88483515c80e49e6"
server_key_version: "encryptionkey@0"
I20260809 16:15:25.280151 19013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:25.280805 19033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:25.280956 19013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:25.281044 19013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "7f48f391ba134f39a5863a6edd1ba26e"
format_stamp: "Formatted at 2026-08-09 16:15:25 on dist-test-slave-fgb3"
server_key: "0afe0e40c89ddf1221f5cb408a180d40"
server_key_iv: "d7a83605da93230e88483515c80e49e6"
server_key_version: "encryptionkey@0"
I20260809 16:15:25.281102 19013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:25.284329 19013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:25.286006 19013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:25.286130 19013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:25.289915 19013 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40859
I20260809 16:15:25.289952 19114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40859 every 8 connection(s)
I20260809 16:15:25.290238 19013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:25.291195 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:25.293465 19119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e: Bootstrap starting.
I20260809 16:15:25.294011 19119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:25.294261 19119 log.cc:824] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:25.294916 19119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e: No bootstrap required, opened a new log
I20260809 16:15:25.296474 19119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f48f391ba134f39a5863a6edd1ba26e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40859 } }
I20260809 16:15:25.296605 19119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:25.296638 19119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f48f391ba134f39a5863a6edd1ba26e, State: Initialized, Role: FOLLOWER
I20260809 16:15:25.296770 19119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f48f391ba134f39a5863a6edd1ba26e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40859 } }
I20260809 16:15:25.296830 19119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:25.296852 19119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:25.296895 19119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:25.297387 19119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f48f391ba134f39a5863a6edd1ba26e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40859 } }
I20260809 16:15:25.297477 19119 leader_election.cc:302] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f48f391ba134f39a5863a6edd1ba26e; no voters:
I20260809 16:15:25.297622 19119 leader_election.cc:288] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:25.297662 19125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:25.297827 19125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [term 1 LEADER]: Becoming Leader. State: Replica: 7f48f391ba134f39a5863a6edd1ba26e, State: Running, Role: LEADER
I20260809 16:15:25.297845 19119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:25.297923 19125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f48f391ba134f39a5863a6edd1ba26e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40859 } }
I20260809 16:15:25.298169 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19013
I20260809 16:15:25.298281 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:25.298502 17282 external_mini_cluster.cc:1428] Setting key 20d4246ae2b7f5380bdfe16aa032276a
I20260809 16:15:25.299223 19130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f48f391ba134f39a5863a6edd1ba26e. Latest consensus state: current_term: 1 leader_uuid: "7f48f391ba134f39a5863a6edd1ba26e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f48f391ba134f39a5863a6edd1ba26e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40859 } } }
I20260809 16:15:25.299374 19130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:25.299245 19129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f48f391ba134f39a5863a6edd1ba26e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f48f391ba134f39a5863a6edd1ba26e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40859 } } }
I20260809 16:15:25.299535 19129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e [sys.catalog]: This master's current role is: LEADER
W20260809 16:15:25.299660 19152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:15:25.299718 19152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:25.300369 19155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:25.300556 19155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:25.300853 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:25.298926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:43607 (local address 127.0.0.1:40859)
0809 16:15:25.299243 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:25.299252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:25.299283 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:15:25.299395 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:25.299402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:25.299434 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:25.299851 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:25.300386 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:25.300390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:25.300390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:25.300394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:25.300433 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:25.300447 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:25.300543 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:25.300634 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:15:25.300678 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":72,"threads_started":1}
I20260809 16:15:25.301870 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:25.303153 19155 catalog_manager.cc:1347] Generated new cluster ID: b236741c286847e593036c6358ffd69e
I20260809 16:15:25.303206 19155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:25.304327 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:25.303412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:46499 (local address 127.0.0.1:40859)
0809 16:15:25.303447 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:25.303449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:25.303498 (+ 49us) server_negotiation.cc:407] Connection header received
0809 16:15:25.303557 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:25.303558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:25.303563 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:25.303627 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:25.304129 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:25.304131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:25.304131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:25.304135 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:25.304151 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:25.304156 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:25.304209 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:25.304255 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 16:15:25.304268 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:25.332166 19155 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:25.332343 19155 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:25.335892 19155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f48f391ba134f39a5863a6edd1ba26e: Generated new TSK 0
I20260809 16:15:25.336040 19155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:25.337353 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19013
2026-08-09T16:15:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:25 dist-test-slave-fgb3 krb5kdc[19171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:25 dist-test-slave-fgb3 krb5kdc[19171](info): set up 2 sockets
Aug 09 16:15:25 dist-test-slave-fgb3 krb5kdc[19171](info): commencing operation
krb5kdc: starting...
W20260809 16:15:26.478420 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:26 dist-test-slave-fgb3 krb5kdc[19171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:26.597510 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41831
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:33113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:41831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:26.679618 19194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:26.679838 19194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:26.679869 19194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:26.681002 19194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:26.681042 19194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:26.681061 19194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:26.681075 19194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:26.681089 19194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:26.682580 19194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:33113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:41831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:26.682828 19194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:26.683037 19194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:26.687767 19208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:26.687844 19204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:26.687975 19194 server_base.cc:1034] running on GCE node
W20260809 16:15:26.688000 19206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:26.688225 19194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:26.688500 19194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:26.689630 19194 hybrid_clock.cc:648] HybridClock initialized: now 1786292126689621 us; error 29 us; skew 500 ppm
Aug 09 16:15:26 dist-test-slave-fgb3 krb5kdc[19171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292126, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:26.694967 19194 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:26.695364 19194 webserver.cc:460] Webserver started at http://127.16.224.190:36065/ using document root <none> and password file <none>
I20260809 16:15:26.695596 19194 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:26.695647 19194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:26.695767 19194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:26.696786 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "1ede9d922225451b935328b2eaefeda7"
format_stamp: "Formatted at 2026-08-09 16:15:26 on dist-test-slave-fgb3"
server_key: "e626fd4a414cd4fb8ebf28abc94f0f25"
server_key_iv: "04c3d21b9ba21fa0b414fe9a745cc8da"
server_key_version: "encryptionkey@0"
I20260809 16:15:26.697120 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "1ede9d922225451b935328b2eaefeda7"
format_stamp: "Formatted at 2026-08-09 16:15:26 on dist-test-slave-fgb3"
server_key: "e626fd4a414cd4fb8ebf28abc94f0f25"
server_key_iv: "04c3d21b9ba21fa0b414fe9a745cc8da"
server_key_version: "encryptionkey@0"
I20260809 16:15:26.698395 19194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:15:26.699072 19222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:26.699216 19194 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:26.699301 19194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "1ede9d922225451b935328b2eaefeda7"
format_stamp: "Formatted at 2026-08-09 16:15:26 on dist-test-slave-fgb3"
server_key: "e626fd4a414cd4fb8ebf28abc94f0f25"
server_key_iv: "04c3d21b9ba21fa0b414fe9a745cc8da"
server_key_version: "encryptionkey@0"
I20260809 16:15:26.699368 19194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:26.717167 19194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:26.719312 19194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:26.719465 19194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:26.723232 19194 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:41831
I20260809 16:15:26.723270 19305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:41831 every 8 connection(s)
I20260809 16:15:26.723599 19194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:26.724417 19306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:26.726709 19306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7: Bootstrap starting.
I20260809 16:15:26.727272 19306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:26.727687 19306 log.cc:824] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:26.728377 19306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7: No bootstrap required, opened a new log
I20260809 16:15:26.729959 19306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede9d922225451b935328b2eaefeda7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41831 } }
I20260809 16:15:26.730095 19306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:26.730125 19306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ede9d922225451b935328b2eaefeda7, State: Initialized, Role: FOLLOWER
I20260809 16:15:26.730259 19306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede9d922225451b935328b2eaefeda7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41831 } }
I20260809 16:15:26.730343 19306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:26.730378 19306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:26.730412 19306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:26.731177 19306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede9d922225451b935328b2eaefeda7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41831 } }
I20260809 16:15:26.731266 19306 leader_election.cc:302] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ede9d922225451b935328b2eaefeda7; no voters:
I20260809 16:15:26.731382 19306 leader_election.cc:288] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:26.731426 19309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:26.731556 19309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [term 1 LEADER]: Becoming Leader. State: Replica: 1ede9d922225451b935328b2eaefeda7, State: Running, Role: LEADER
I20260809 16:15:26.731638 19306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:26.731647 19309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede9d922225451b935328b2eaefeda7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41831 } }
I20260809 16:15:26.732034 19310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ede9d922225451b935328b2eaefeda7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede9d922225451b935328b2eaefeda7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41831 } } }
I20260809 16:15:26.732050 19311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ede9d922225451b935328b2eaefeda7. Latest consensus state: current_term: 1 leader_uuid: "1ede9d922225451b935328b2eaefeda7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede9d922225451b935328b2eaefeda7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41831 } } }
I20260809 16:15:26.732139 19311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:26.732136 19310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:26.732355 19321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:26.732798 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19194
I20260809 16:15:26.732846 19321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:26.732918 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:26.733088 17282 external_mini_cluster.cc:1428] Setting key cc0cd7606b66fed1a4950281e365250f
I20260809 16:15:26.738129 19321 catalog_manager.cc:1347] Generated new cluster ID: b016fdef937a4345b1ce727454b60b92
I20260809 16:15:26.738181 19321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:15:26 dist-test-slave-fgb3 krb5kdc[19171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:26.743319 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:26.733523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35409 (local address 127.16.224.190:41831)
0809 16:15:26.733708 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:26.733718 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:26.733753 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:15:26.733893 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:26.733900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:26.733960 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:26.734071 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:26.734717 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:26.735606 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:26.736341 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:26.736769 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:26.736864 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:26.739618 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:26.739623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:26.739624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:26.739636 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:26.741605 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:26.742020 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:26.742023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:26.742024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:26.742159 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:26.742378 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:26.742379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:26.742379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:26.742702 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:26.742877 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:26.742969 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 16:15:26.743073 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260809 16:15:26.744525 19321 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:26.744518 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:26.745157 19321 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:26.752990 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:26.745951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58557 (local address 127.16.224.190:41831)
0809 16:15:26.745984 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:26.745986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:26.746101 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:15:26.746135 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:26.746136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:26.746146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:26.746201 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:26.746782 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:26.747598 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:26.748562 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:26.748906 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:26.748969 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:26.750307 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:26.750348 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:26.750348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:26.750355 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:26.751860 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:26.752244 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:26.752246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:26.752246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:26.752309 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:26.752500 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:26.752501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:26.752501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:26.752628 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:26.752741 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:26.752830 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:15:26.752893 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:15:26.757282 19321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ede9d922225451b935328b2eaefeda7: Generated new TSK 0
I20260809 16:15:26.757522 19321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:26.760926 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19194
W20260809 16:15:26.779013 19169 connection.cc:535] client connection to 127.16.224.190:41831 recv error: Network error: failed to read from TLS socket (remote: 127.16.224.190:41831): Connection reset by peer (error 104)
2026-08-09T16:15:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:26 dist-test-slave-fgb3 krb5kdc[19357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:26 dist-test-slave-fgb3 krb5kdc[19357](info): set up 2 sockets
Aug 09 16:15:26 dist-test-slave-fgb3 krb5kdc[19357](info): commencing operation
krb5kdc: starting...
W20260809 16:15:27.912037 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:27 dist-test-slave-fgb3 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:28.028422 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42403
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:34401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:42403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:28.110272 19382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:28.110505 19382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:28.110527 19382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:28.111652 19382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:28.111694 19382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:28.111707 19382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:28.111718 19382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:28.111728 19382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:28.113196 19382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:34401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:42403
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:42403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:28.113435 19382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:28.113679 19382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:28.118294 19392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:28.118340 19393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:28.118289 19396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:28.118651 19382 server_base.cc:1034] running on GCE node
I20260809 16:15:28.118871 19382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:28.119129 19382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:28.120261 19382 hybrid_clock.cc:648] HybridClock initialized: now 1786292128120243 us; error 35 us; skew 500 ppm
Aug 09 16:15:28 dist-test-slave-fgb3 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292128, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:28.122653 19382 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:28.123039 19382 webserver.cc:460] Webserver started at http://127.16.224.190:40591/ using document root <none> and password file <none>
I20260809 16:15:28.123243 19382 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:28.123287 19382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:28.123376 19382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:28.124372 19382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "59fa99949fa340d4a588b98d4770c1dd"
format_stamp: "Formatted at 2026-08-09 16:15:28 on dist-test-slave-fgb3"
server_key: "68b1f1554a46a2e6d906cc39e89470b1"
server_key_iv: "6c18e55bef3ea6c098d7b5f451548c82"
server_key_version: "encryptionkey@0"
I20260809 16:15:28.124665 19382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "59fa99949fa340d4a588b98d4770c1dd"
format_stamp: "Formatted at 2026-08-09 16:15:28 on dist-test-slave-fgb3"
server_key: "68b1f1554a46a2e6d906cc39e89470b1"
server_key_iv: "6c18e55bef3ea6c098d7b5f451548c82"
server_key_version: "encryptionkey@0"
I20260809 16:15:28.125821 19382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:15:28.126464 19403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:28.126619 19382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:28.126704 19382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "59fa99949fa340d4a588b98d4770c1dd"
format_stamp: "Formatted at 2026-08-09 16:15:28 on dist-test-slave-fgb3"
server_key: "68b1f1554a46a2e6d906cc39e89470b1"
server_key_iv: "6c18e55bef3ea6c098d7b5f451548c82"
server_key_version: "encryptionkey@0"
I20260809 16:15:28.126760 19382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:28.156212 19382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:28.158020 19382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:28.158138 19382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:28.162071 19382 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:42403
I20260809 16:15:28.162127 19481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:42403 every 8 connection(s)
I20260809 16:15:28.162467 19382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:28.163347 19482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:28.163851 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19382
I20260809 16:15:28.163962 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:28.164136 17282 external_mini_cluster.cc:1428] Setting key 429bdb7f606c88ccf32ce613c2be5a9b
I20260809 16:15:28.165870 19482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd: Bootstrap starting.
I20260809 16:15:28.166484 19482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:28.166738 19482 log.cc:824] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:28.167371 19482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd: No bootstrap required, opened a new log
I20260809 16:15:28.169191 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:28.164630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34213 (local address 127.16.224.190:42403)
0809 16:15:28.164810 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:28.164818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:28.164842 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:15:28.165240 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:28.165247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:28.165292 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:28.165415 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:28.165646 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:28.166611 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:28.167397 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:28.167842 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:28.167948 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:28.168637 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:28.168642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:28.168643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:28.168795 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:28.168824 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:28.168834 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:28.168902 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:28.168993 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:15:28.169021 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260809 16:15:28.169363 19482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fa99949fa340d4a588b98d4770c1dd" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42403 } }
I20260809 16:15:28.169540 19482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:28.169615 19482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59fa99949fa340d4a588b98d4770c1dd, State: Initialized, Role: FOLLOWER
I20260809 16:15:28.169736 19482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fa99949fa340d4a588b98d4770c1dd" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42403 } }
I20260809 16:15:28.169839 19482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:28.169879 19482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:28.169930 19482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:28.170186 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:28.170614 19482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fa99949fa340d4a588b98d4770c1dd" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42403 } }
I20260809 16:15:28.170729 19482 leader_election.cc:302] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59fa99949fa340d4a588b98d4770c1dd; no voters:
I20260809 16:15:28.170902 19482 leader_election.cc:288] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:28.170946 19497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:28.171099 19497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [term 1 LEADER]: Becoming Leader. State: Replica: 59fa99949fa340d4a588b98d4770c1dd, State: Running, Role: LEADER
I20260809 16:15:28.171149 19482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:28.171193 19497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fa99949fa340d4a588b98d4770c1dd" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42403 } }
I20260809 16:15:28.171506 19497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59fa99949fa340d4a588b98d4770c1dd. Latest consensus state: current_term: 1 leader_uuid: "59fa99949fa340d4a588b98d4770c1dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fa99949fa340d4a588b98d4770c1dd" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42403 } } }
I20260809 16:15:28.171581 19497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:28.171792 19512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:28.171856 19500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59fa99949fa340d4a588b98d4770c1dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fa99949fa340d4a588b98d4770c1dd" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 42403 } } }
I20260809 16:15:28.171924 19500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:28.172610 19512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:28.174182 19512 catalog_manager.cc:1347] Generated new cluster ID: f5d1eb2d189b4ee6a639bfae48f5e341
I20260809 16:15:28.174235 19512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:28.176317 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:28.172307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50651 (local address 127.16.224.190:42403)
0809 16:15:28.172338 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:28.172340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:28.173182 (+ 842us) server_negotiation.cc:407] Connection header received
0809 16:15:28.173227 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:28.173228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:28.173238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:28.173291 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:28.173433 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:28.174033 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:28.175045 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:28.175346 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:28.175391 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:28.176083 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:28.176084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:28.176085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:28.176089 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:28.176105 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:28.176110 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:28.176168 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:28.176236 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:15:28.176246 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 16:15:28.189301 19512 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:28.189864 19512 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:28.199219 19512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59fa99949fa340d4a588b98d4770c1dd: Generated new TSK 0
I20260809 16:15:28.199429 19512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:28.203801 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19382
W20260809 16:15:28.219110 19352 connection.cc:535] client connection to 127.16.224.190:42403 recv error: Network error: failed to read from TLS socket (remote: 127.16.224.190:42403): Connection reset by peer (error 104)
2026-08-09T16:15:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:28 dist-test-slave-fgb3 krb5kdc[19541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:28 dist-test-slave-fgb3 krb5kdc[19541](info): set up 2 sockets
Aug 09 16:15:28 dist-test-slave-fgb3 krb5kdc[19541](info): commencing operation
krb5kdc: starting...
W20260809 16:15:29.351837 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:29 dist-test-slave-fgb3 krb5kdc[19541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:29.474558 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:32969
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:32969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:29.556450 19564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:29.556671 19564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:29.556704 19564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:29.557844 19564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:29.557888 19564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:29.557906 19564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:29.557921 19564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:29.557937 19564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:29.559494 19564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:32969
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:32969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:29.559751 19564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:29.559968 19564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:29.564742 19573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:29.564774 19570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:29.564766 19571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:29.565073 19564 server_base.cc:1034] running on GCE node
I20260809 16:15:29.565327 19564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:29.565613 19564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:29.566749 19564 hybrid_clock.cc:648] HybridClock initialized: now 1786292129566734 us; error 26 us; skew 500 ppm
Aug 09 16:15:29 dist-test-slave-fgb3 krb5kdc[19541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292129, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:29.569144 19564 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:29.569612 19564 webserver.cc:460] Webserver started at http://127.16.224.190:33461/ using document root <none> and password file <none>
I20260809 16:15:29.569813 19564 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:29.569861 19564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:29.569988 19564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:29.571110 19564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "6105db9e38fb497780a93a162cbc62be"
format_stamp: "Formatted at 2026-08-09 16:15:29 on dist-test-slave-fgb3"
server_key: "ab28e4e1cb7027cd0f44c149c7e26a40"
server_key_iv: "c4987fa9a5cf437cc04a9938201c7454"
server_key_version: "encryptionkey@0"
I20260809 16:15:29.571461 19564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "6105db9e38fb497780a93a162cbc62be"
format_stamp: "Formatted at 2026-08-09 16:15:29 on dist-test-slave-fgb3"
server_key: "ab28e4e1cb7027cd0f44c149c7e26a40"
server_key_iv: "c4987fa9a5cf437cc04a9938201c7454"
server_key_version: "encryptionkey@0"
I20260809 16:15:29.572702 19564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:15:29.573397 19584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:29.573553 19564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:29.573645 19564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "6105db9e38fb497780a93a162cbc62be"
format_stamp: "Formatted at 2026-08-09 16:15:29 on dist-test-slave-fgb3"
server_key: "ab28e4e1cb7027cd0f44c149c7e26a40"
server_key_iv: "c4987fa9a5cf437cc04a9938201c7454"
server_key_version: "encryptionkey@0"
I20260809 16:15:29.573704 19564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:29.576943 19564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:29.578681 19564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:29.578809 19564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:29.582593 19564 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:32969
I20260809 16:15:29.582623 19669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:32969 every 8 connection(s)
I20260809 16:15:29.582914 19564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:29.583842 19670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:29.586009 19670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be: Bootstrap starting.
I20260809 16:15:29.586588 19670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:29.586812 19670 log.cc:824] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:29.587437 19670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be: No bootstrap required, opened a new log
I20260809 16:15:29.588971 19670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6105db9e38fb497780a93a162cbc62be" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32969 } }
I20260809 16:15:29.589108 19670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:29.589129 19670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6105db9e38fb497780a93a162cbc62be, State: Initialized, Role: FOLLOWER
I20260809 16:15:29.589231 19670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6105db9e38fb497780a93a162cbc62be" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32969 } }
I20260809 16:15:29.589279 19670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:29.589299 19670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:29.589329 19670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:29.589680 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19564
I20260809 16:15:29.589818 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:29.590029 17282 external_mini_cluster.cc:1428] Setting key 8102cecbe15a0de7256eeb63edc8406a
I20260809 16:15:29.590381 19670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6105db9e38fb497780a93a162cbc62be" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32969 } }
I20260809 16:15:29.590492 19670 leader_election.cc:302] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6105db9e38fb497780a93a162cbc62be; no voters:
I20260809 16:15:29.590654 19670 leader_election.cc:288] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:29.590818 19680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:29.590909 19670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:29.590984 19680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [term 1 LEADER]: Becoming Leader. State: Replica: 6105db9e38fb497780a93a162cbc62be, State: Running, Role: LEADER
I20260809 16:15:29.591123 19680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6105db9e38fb497780a93a162cbc62be" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32969 } }
I20260809 16:15:29.591527 19678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6105db9e38fb497780a93a162cbc62be. Latest consensus state: current_term: 1 leader_uuid: "6105db9e38fb497780a93a162cbc62be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6105db9e38fb497780a93a162cbc62be" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32969 } } }
I20260809 16:15:29.591711 19678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:29.591524 19681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6105db9e38fb497780a93a162cbc62be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6105db9e38fb497780a93a162cbc62be" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 32969 } } }
I20260809 16:15:29.592343 19681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:29.592377 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:29.590570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50925 (local address 127.16.224.190:32969)
0809 16:15:29.590783 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:29.590792 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:29.590825 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:15:29.590953 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:29.590961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:29.590991 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:29.591148 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:29.591825 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:29.591830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:29.591830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:29.591834 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:29.591879 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:29.591894 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:29.591951 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:29.592181 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 16:15:29.592216 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":65,"threads_started":1}
I20260809 16:15:29.592528 19690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:29.593044 19690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:29.593379 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:29.594642 19690 catalog_manager.cc:1347] Generated new cluster ID: 93ccc65f31c445efa8b587e62d3d97ee
I20260809 16:15:29.594697 19690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:29.599542 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:29.598549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35525 (local address 127.16.224.190:32969)
0809 16:15:29.598583 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:29.598585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:29.598709 (+ 124us) server_negotiation.cc:407] Connection header received
0809 16:15:29.598765 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:29.598767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:29.598773 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:29.598817 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:29.599320 (+ 503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:29.599321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:29.599321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:29.599325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:29.599341 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:29.599345 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:29.599393 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:29.599451 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 16:15:29.599465 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:15:29.610183 19690 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:29.610385 19690 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:29.624007 19690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6105db9e38fb497780a93a162cbc62be: Generated new TSK 0
I20260809 16:15:29.624192 19690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:29.632859 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19564
2026-08-09T16:15:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:29 dist-test-slave-fgb3 krb5kdc[19715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:29 dist-test-slave-fgb3 krb5kdc[19715](info): set up 2 sockets
Aug 09 16:15:29 dist-test-slave-fgb3 krb5kdc[19715](info): commencing operation
krb5kdc: starting...
W20260809 16:15:30.780902 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:30 dist-test-slave-fgb3 krb5kdc[19715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:30.900751 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:37475
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:44539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:37475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:30.981962 19741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:30.982163 19741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:30.982185 19741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:30.983330 19741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:30.983373 19741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:30.983386 19741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:30.983397 19741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:30.983408 19741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:30.984895 19741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:44539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:37475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:37475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:30.985134 19741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:30.985347 19741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:30.990031 19747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:30.990036 19752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:30.990051 19741 server_base.cc:1034] running on GCE node
W20260809 16:15:30.990051 19749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:30.990507 19741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:30.990780 19741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:30.991919 19741 hybrid_clock.cc:648] HybridClock initialized: now 1786292130991902 us; error 27 us; skew 500 ppm
Aug 09 16:15:30 dist-test-slave-fgb3 krb5kdc[19715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292130, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:30.994247 19741 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:30.994688 19741 webserver.cc:460] Webserver started at http://127.16.224.190:39425/ using document root <none> and password file <none>
I20260809 16:15:30.994897 19741 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:30.994939 19741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:30.995054 19741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:30.996054 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "873080b30bd049a5aaeb1d54120809a7"
format_stamp: "Formatted at 2026-08-09 16:15:30 on dist-test-slave-fgb3"
server_key: "8f5661b90de121f2b70cf867bb076ed4"
server_key_iv: "f09676264c11eb0635d4cbc13b861b4c"
server_key_version: "encryptionkey@0"
I20260809 16:15:30.996379 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "873080b30bd049a5aaeb1d54120809a7"
format_stamp: "Formatted at 2026-08-09 16:15:30 on dist-test-slave-fgb3"
server_key: "8f5661b90de121f2b70cf867bb076ed4"
server_key_iv: "f09676264c11eb0635d4cbc13b861b4c"
server_key_version: "encryptionkey@0"
I20260809 16:15:30.997629 19741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:30.998376 19763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:30.998543 19741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:15:30.998625 19741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "873080b30bd049a5aaeb1d54120809a7"
format_stamp: "Formatted at 2026-08-09 16:15:30 on dist-test-slave-fgb3"
server_key: "8f5661b90de121f2b70cf867bb076ed4"
server_key_iv: "f09676264c11eb0635d4cbc13b861b4c"
server_key_version: "encryptionkey@0"
I20260809 16:15:30.998694 19741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:31.017660 19741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:31.019989 19741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:31.020128 19741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:31.024042 19741 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:37475
I20260809 16:15:31.024080 19853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:37475 every 8 connection(s)
I20260809 16:15:31.024395 19741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:31.025220 19854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:31.025728 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19741
I20260809 16:15:31.025859 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:31.026047 17282 external_mini_cluster.cc:1428] Setting key a57c4b9327cb0bd89d26d24d912d44fe
I20260809 16:15:31.028260 19854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7: Bootstrap starting.
I20260809 16:15:31.028898 19854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:31.029183 19854 log.cc:824] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:31.029930 19854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7: No bootstrap required, opened a new log
I20260809 16:15:31.032341 19854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873080b30bd049a5aaeb1d54120809a7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37475 } }
Aug 09 16:15:31 dist-test-slave-fgb3 krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:31.032474 19854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:31.032505 19854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 873080b30bd049a5aaeb1d54120809a7, State: Initialized, Role: FOLLOWER
I20260809 16:15:31.032660 19854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873080b30bd049a5aaeb1d54120809a7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37475 } }
I20260809 16:15:31.032729 19854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:31.032750 19854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:31.032783 19854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:31.033288 19854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873080b30bd049a5aaeb1d54120809a7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37475 } }
I20260809 16:15:31.033391 19854 leader_election.cc:302] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 873080b30bd049a5aaeb1d54120809a7; no voters:
I20260809 16:15:31.033540 19854 leader_election.cc:288] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:31.033591 19866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:31.033701 19866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [term 1 LEADER]: Becoming Leader. State: Replica: 873080b30bd049a5aaeb1d54120809a7, State: Running, Role: LEADER
I20260809 16:15:31.033804 19854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:31.033874 19866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873080b30bd049a5aaeb1d54120809a7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37475 } }
I20260809 16:15:31.035053 19869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 873080b30bd049a5aaeb1d54120809a7. Latest consensus state: current_term: 1 leader_uuid: "873080b30bd049a5aaeb1d54120809a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873080b30bd049a5aaeb1d54120809a7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37475 } } }
I20260809 16:15:31.035240 19869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:31.035284 19868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "873080b30bd049a5aaeb1d54120809a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873080b30bd049a5aaeb1d54120809a7" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 37475 } } }
I20260809 16:15:31.035355 19868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7 [sys.catalog]: This master's current role is: LEADER
W20260809 16:15:31.035538 19888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:15:31.035593 19888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:31.035652 19893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:31.035804 19893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:31.036482 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:31.026567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57081 (local address 127.16.224.190:37475)
0809 16:15:31.026717 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:31.026724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:31.026749 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:15:31.027159 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:31.027166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:31.027211 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:31.027332 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:31.028142 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:31.029038 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:31.029962 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:31.030425 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:31.030504 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:31.032994 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:31.032998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:31.032998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:31.033009 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:31.034827 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:31.035233 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:31.035236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:31.035237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:31.035369 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:31.035578 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:31.035579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:31.035580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:31.035891 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:31.036048 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:31.036154 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 16:15:31.036289 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260809 16:15:31.037369 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:31.037494 19893 catalog_manager.cc:1347] Generated new cluster ID: 45b0c297161c4ca2ba99992a8f385176
I20260809 16:15:31.037536 19893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:31.048903 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:31.042396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58703 (local address 127.16.224.190:37475)
0809 16:15:31.042451 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:31.042453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:31.042462 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:15:31.042482 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:31.042483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:31.042494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:31.042555 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:31.043112 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:31.043851 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:31.044790 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:31.045110 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:31.045179 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:31.046416 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:31.046418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:31.046418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:31.046425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:31.047870 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:31.048209 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:31.048210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:31.048210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:31.048266 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:31.048421 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:31.048421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:31.048422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:31.048542 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:31.048625 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:31.048729 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:15:31.048801 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:15:31.054622 19893 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:31.055338 19893 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:31.072650 19893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 873080b30bd049a5aaeb1d54120809a7: Generated new TSK 0
I20260809 16:15:31.072903 19893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:31.079592 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19741
2026-08-09T16:15:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:31 dist-test-slave-fgb3 krb5kdc[19908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:31 dist-test-slave-fgb3 krb5kdc[19908](info): set up 2 sockets
Aug 09 16:15:31 dist-test-slave-fgb3 krb5kdc[19908](info): commencing operation
krb5kdc: starting...
W20260809 16:15:32.233381 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:32 dist-test-slave-fgb3 krb5kdc[19908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:32.346902 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41523
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:41523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:32.426127 19931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:32.426343 19931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:32.426386 19931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:32.427480 19931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:32.427518 19931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:32.427537 19931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:32.427554 19931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:32.427569 19931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:32.429003 19931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:41523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:41523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:32.429242 19931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:32.429454 19931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:15:32.433941 19931 server_base.cc:1034] running on GCE node
W20260809 16:15:32.433910 19941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:32.433905 19940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:32.433892 19943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:32.434301 19931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:32.434608 19931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:32.435745 19931 hybrid_clock.cc:648] HybridClock initialized: now 1786292132435731 us; error 33 us; skew 500 ppm
Aug 09 16:15:32 dist-test-slave-fgb3 krb5kdc[19908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292132, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:32.441092 19931 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:32.441543 19931 webserver.cc:460] Webserver started at http://127.16.224.190:36917/ using document root <none> and password file <none>
I20260809 16:15:32.441769 19931 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:32.441818 19931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:32.441946 19931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:32.442999 19931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "1bf892cae65646cd81d5e121e3899bbe"
format_stamp: "Formatted at 2026-08-09 16:15:32 on dist-test-slave-fgb3"
server_key: "f295c75cf8eae121bbcb7a084eafc1bf"
server_key_iv: "325402c0091bc95c4497b042c6a3c15b"
server_key_version: "encryptionkey@0"
I20260809 16:15:32.443331 19931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "1bf892cae65646cd81d5e121e3899bbe"
format_stamp: "Formatted at 2026-08-09 16:15:32 on dist-test-slave-fgb3"
server_key: "f295c75cf8eae121bbcb7a084eafc1bf"
server_key_iv: "325402c0091bc95c4497b042c6a3c15b"
server_key_version: "encryptionkey@0"
I20260809 16:15:32.444558 19931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:32.445194 19952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:32.445339 19931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:32.445433 19931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "1bf892cae65646cd81d5e121e3899bbe"
format_stamp: "Formatted at 2026-08-09 16:15:32 on dist-test-slave-fgb3"
server_key: "f295c75cf8eae121bbcb7a084eafc1bf"
server_key_iv: "325402c0091bc95c4497b042c6a3c15b"
server_key_version: "encryptionkey@0"
I20260809 16:15:32.445493 19931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:32.464478 19931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:32.466329 19931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:32.466485 19931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:32.471120 19931 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:41523
I20260809 16:15:32.471159 20046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:41523 every 8 connection(s)
I20260809 16:15:32.471657 19931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:32.472323 20048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:32.474673 20048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe: Bootstrap starting.
I20260809 16:15:32.475217 20048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:32.475481 20048 log.cc:824] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:32.476125 20048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe: No bootstrap required, opened a new log
I20260809 16:15:32.477676 20048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bf892cae65646cd81d5e121e3899bbe" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41523 } }
I20260809 16:15:32.477815 20048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:32.477847 20048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bf892cae65646cd81d5e121e3899bbe, State: Initialized, Role: FOLLOWER
I20260809 16:15:32.477980 20048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bf892cae65646cd81d5e121e3899bbe" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41523 } }
I20260809 16:15:32.478056 20048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:32.478080 20048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:32.478125 20048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:32.479038 20048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bf892cae65646cd81d5e121e3899bbe" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41523 } }
I20260809 16:15:32.479130 20048 leader_election.cc:302] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bf892cae65646cd81d5e121e3899bbe; no voters:
I20260809 16:15:32.479279 20048 leader_election.cc:288] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:32.479310 20052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:32.479450 20052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [term 1 LEADER]: Becoming Leader. State: Replica: 1bf892cae65646cd81d5e121e3899bbe, State: Running, Role: LEADER
I20260809 16:15:32.479529 20048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:32.479568 20052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bf892cae65646cd81d5e121e3899bbe" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41523 } }
I20260809 16:15:32.479930 20053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bf892cae65646cd81d5e121e3899bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bf892cae65646cd81d5e121e3899bbe" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41523 } } }
I20260809 16:15:32.479959 20054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bf892cae65646cd81d5e121e3899bbe. Latest consensus state: current_term: 1 leader_uuid: "1bf892cae65646cd81d5e121e3899bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bf892cae65646cd81d5e121e3899bbe" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 41523 } } }
I20260809 16:15:32.480064 20053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:32.480075 20054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:32.480499 20064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:32.480948 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 19931
I20260809 16:15:32.481019 20064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:32.481060 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:32.481226 17282 external_mini_cluster.cc:1428] Setting key d8bfed76d2c0cb0b91e150226485eb95
I20260809 16:15:32.482775 20064 catalog_manager.cc:1347] Generated new cluster ID: fa4f85de97b245f1a47bc56f05db8b8d
I20260809 16:15:32.482831 20064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:32.490057 20064 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:32.490726 20064 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:32.495172 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:32.481635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49439 (local address 127.16.224.190:41523)
0809 16:15:32.489966 (+ 8331us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:32.489975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:32.490012 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:15:32.490132 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:32.490139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:32.490208 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:32.490388 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:32.490626 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:32.491997 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:32.493143 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:32.493727 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:32.493816 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:32.494508 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:32.494512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:32.494512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:32.494711 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:32.494747 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:32.494761 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:32.494821 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:32.494893 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 16:15:32.494962 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8282,"thread_start_us":76,"threads_started":1}
I20260809 16:15:32.496045 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:32.499446 20064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bf892cae65646cd81d5e121e3899bbe: Generated new TSK 0
I20260809 16:15:32.499611 20064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:32.500851 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:32.497525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50703 (local address 127.16.224.190:41523)
0809 16:15:32.497563 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:32.497565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:32.497636 (+ 71us) server_negotiation.cc:407] Connection header received
0809 16:15:32.497688 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:32.497689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:32.497698 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:32.497760 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:32.497877 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:32.498668 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:32.499614 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:32.499952 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:32.500036 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:32.500593 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:32.500594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:32.500594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:32.500601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:32.500616 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:32.500621 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:32.500692 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:32.500738 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 16:15:32.500774 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:15:32.502871 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 19931
2026-08-09T16:15:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:32 dist-test-slave-fgb3 krb5kdc[20093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:32 dist-test-slave-fgb3 krb5kdc[20093](info): set up 2 sockets
Aug 09 16:15:32 dist-test-slave-fgb3 krb5kdc[20093](info): commencing operation
krb5kdc: starting...
W20260809 16:15:33.654654 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:33 dist-test-slave-fgb3 krb5kdc[20093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:33.773221 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46499
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:34839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:46499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:33.854084 20112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:33.854290 20112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:33.854329 20112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:33.855451 20112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:33.855491 20112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:33.855504 20112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:33.855515 20112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:33.855525 20112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:33.856968 20112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:34839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:46499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:46499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:33.857165 20112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:33.857337 20112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:33.862010 20122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:33.862015 20120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:33.862080 20112 server_base.cc:1034] running on GCE node
W20260809 16:15:33.862010 20119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:33.862528 20112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:33.862807 20112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:33.863947 20112 hybrid_clock.cc:648] HybridClock initialized: now 1786292133863929 us; error 30 us; skew 500 ppm
Aug 09 16:15:33 dist-test-slave-fgb3 krb5kdc[20093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292133, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:33.866389 20112 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:33.866816 20112 webserver.cc:460] Webserver started at http://127.16.224.190:46015/ using document root <none> and password file <none>
I20260809 16:15:33.867012 20112 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:33.867055 20112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:33.867173 20112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:33.868269 20112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "7a3c42712cb146cca7acc3dcba0a1aef"
format_stamp: "Formatted at 2026-08-09 16:15:33 on dist-test-slave-fgb3"
server_key: "73a35fb5b4e3bc1344f5d1b50766f471"
server_key_iv: "a202c8fe121d7f8f23d31fccbfa7a53a"
server_key_version: "encryptionkey@0"
I20260809 16:15:33.868607 20112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "7a3c42712cb146cca7acc3dcba0a1aef"
format_stamp: "Formatted at 2026-08-09 16:15:33 on dist-test-slave-fgb3"
server_key: "73a35fb5b4e3bc1344f5d1b50766f471"
server_key_iv: "a202c8fe121d7f8f23d31fccbfa7a53a"
server_key_version: "encryptionkey@0"
I20260809 16:15:33.869810 20112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:15:33.870452 20137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:33.870604 20112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:33.870692 20112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "7a3c42712cb146cca7acc3dcba0a1aef"
format_stamp: "Formatted at 2026-08-09 16:15:33 on dist-test-slave-fgb3"
server_key: "73a35fb5b4e3bc1344f5d1b50766f471"
server_key_iv: "a202c8fe121d7f8f23d31fccbfa7a53a"
server_key_version: "encryptionkey@0"
I20260809 16:15:33.870759 20112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:33.873973 20112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:33.875984 20112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:33.876109 20112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:33.879819 20112 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:46499
I20260809 16:15:33.879851 20234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:46499 every 8 connection(s)
I20260809 16:15:33.880122 20112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:33.881094 20236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:33.883327 20236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef: Bootstrap starting.
I20260809 16:15:33.883852 20236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:33.884109 20236 log.cc:824] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:33.884755 20236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef: No bootstrap required, opened a new log
I20260809 16:15:33.886273 20236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46499 } }
I20260809 16:15:33.886441 20236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:15:33.886468 20236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3c42712cb146cca7acc3dcba0a1aef, State: Initialized, Role: FOLLOWER
I20260809 16:15:33.886607 20236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46499 } }
I20260809 16:15:33.886683 20236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:15:33.886716 20236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:15:33.886761 20236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:33.887544 20236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46499 } }
I20260809 16:15:33.887629 20236 leader_election.cc:302] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a3c42712cb146cca7acc3dcba0a1aef; no voters:
I20260809 16:15:33.887776 20236 leader_election.cc:288] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:15:33.887815 20241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:33.887948 20241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [term 1 LEADER]: Becoming Leader. State: Replica: 7a3c42712cb146cca7acc3dcba0a1aef, State: Running, Role: LEADER
I20260809 16:15:33.888018 20236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:33.888037 20241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46499 } }
I20260809 16:15:33.888417 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20112
I20260809 16:15:33.888414 20244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a3c42712cb146cca7acc3dcba0a1aef. Latest consensus state: current_term: 1 leader_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46499 } } }
I20260809 16:15:33.888434 20242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3c42712cb146cca7acc3dcba0a1aef" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 46499 } } }
I20260809 16:15:33.888484 20244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:33.888517 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:33.888530 20242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:33.888707 17282 external_mini_cluster.cc:1428] Setting key 5989759f9ec996396edffb9f2d4cde5b
I20260809 16:15:33.888723 20252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:33.889433 20252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:33.891469 20252 catalog_manager.cc:1347] Generated new cluster ID: ebc3684a16c64e8ba776c3f177c18b2f
I20260809 16:15:33.891515 20252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:33.892014 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:33.889100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39709 (local address 127.16.224.190:46499)
0809 16:15:33.889261 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:33.889270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:33.889778 (+ 508us) server_negotiation.cc:407] Connection header received
0809 16:15:33.890098 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:33.890105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:33.890136 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:33.890272 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:33.891310 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:33.891314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:33.891314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:33.891362 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:33.891452 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:33.891472 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:33.891580 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:33.891816 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 16:15:33.891870 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260809 16:15:33.892792 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:33.899610 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:33.898682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51833 (local address 127.16.224.190:46499)
0809 16:15:33.898715 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:33.898716 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:33.898814 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:15:33.898871 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:33.898872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:33.898877 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:33.898920 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:33.899411 (+ 491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:33.899412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:33.899412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:15:33.899424 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:33.899438 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:15:33.899442 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:33.899488 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:33.899537 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 16:15:33.899550 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:33.906028 20252 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:33.906183 20252 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:33.911346 20252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a3c42712cb146cca7acc3dcba0a1aef: Generated new TSK 0
I20260809 16:15:33.911485 20252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:33.915737 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20112
2026-08-09T16:15:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1409 ms)
[----------] 18 tests from AuthTokenIssuingTest (25735 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:33 dist-test-slave-fgb3 krb5kdc[20287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:33 dist-test-slave-fgb3 krb5kdc[20287](info): set up 2 sockets
Aug 09 16:15:33 dist-test-slave-fgb3 krb5kdc[20287](info): commencing operation
krb5kdc: starting...
W20260809 16:15:35.057188 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:35.173473 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43159
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:43159,127.16.224.189:33743,127.16.224.188:37167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:35.255326 20315 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:15:35.255527 20315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:35.255543 20315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:35.255560 20315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:35.256636 20315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:35.256669 20315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:35.256682 20315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:35.256692 20315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:35.256706 20315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:35.258172 20315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:43159,127.16.224.189:33743,127.16.224.188:37167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:43159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:35.258414 20315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:35.258598 20315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:35.263087 20327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:35.263101 20324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:35.263139 20322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:35.263365 20315 server_base.cc:1034] running on GCE node
I20260809 16:15:35.263577 20315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:35.263823 20315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:35.264957 20315 hybrid_clock.cc:648] HybridClock initialized: now 1786292135264943 us; error 30 us; skew 500 ppm
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:35.267349 20315 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:35.267728 20315 webserver.cc:460] Webserver started at http://127.16.224.190:37167/ using document root <none> and password file <none>
I20260809 16:15:35.267925 20315 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:35.267964 20315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:35.268059 20315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:35.269086 20315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "3c241757afba431fb10ddcfa600914dc"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "d1a60bd5247111da40ada590c7aff320"
server_key_iv: "65b1cff13be3fd862edde56612d8738b"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.269384 20315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "3c241757afba431fb10ddcfa600914dc"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "d1a60bd5247111da40ada590c7aff320"
server_key_iv: "65b1cff13be3fd862edde56612d8738b"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.270624 20315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:35.271301 20341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:35.271446 20315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:35.271533 20315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "3c241757afba431fb10ddcfa600914dc"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "d1a60bd5247111da40ada590c7aff320"
server_key_iv: "65b1cff13be3fd862edde56612d8738b"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.271595 20315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:35.290079 20315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:35.291853 20315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:35.291998 20315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:35.295945 20315 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:43159
I20260809 16:15:35.295979 20439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:43159 every 8 connection(s)
I20260809 16:15:35.296311 20315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:35.297166 20440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:35.298418 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20315
I20260809 16:15:35.298537 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:35.298715 17282 external_mini_cluster.cc:1428] Setting key fb8c21ff0e5b3bf06a878fbaed85d90a
I20260809 16:15:35.299073 20440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.224.189@KRBTEST.COM; defaulting to no policy
I20260809 16:15:35.308449 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.299830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43159 (local address 127.0.0.1:35348)
0809 16:15:35.300017 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.300059 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.300683 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.300954 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.300956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.301225 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.301377 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.301382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.302405 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.302406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.303131 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.303135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.303593 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.303594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.303659 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.304204 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.304213 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.305534 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.307142 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.307149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.307151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.307326 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.307528 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.307528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.307529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.307611 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.307925 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.307930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.307992 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.308073 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.308289 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":82,"thread_start_us":94,"threads_started":1}
I20260809 16:15:35.308459 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.299866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35348 (local address 127.16.224.190:43159)
0809 16:15:35.300039 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.300048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.300153 (+ 105us) server_negotiation.cc:407] Connection header received
0809 16:15:35.300839 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.300841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.300861 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.300944 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.301473 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.302291 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.303205 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.303520 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.303621 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.305611 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.305619 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.305620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.305626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.307071 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.307394 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.307397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.307397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.307470 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.307663 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.307664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.307664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.307862 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.307941 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.308112 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.308216 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.189@KRBTEST.COM" created.
I20260809 16:15:35.309327 20440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:35.309671 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.309563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60850)
0809 16:15:35.309607 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.309648 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:15:35.309824 20345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
I20260809 16:15:35.310237 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.310143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60856)
0809 16:15:35.310184 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.310214 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 16:15:35.310370 20440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:33743: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab.
Entry for principal kudu/127.16.224.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab.
I20260809 16:15:35.324453 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:33743
--webserver_interface=127.16.224.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab
--principal=kudu/127.16.224.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:43159,127.16.224.189:33743,127.16.224.188:37167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:15:35.359594 20440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } attempt: 1
I20260809 16:15:35.360172 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.360023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60864)
0809 16:15:35.360093 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.360138 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 16:15:35.360538 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.360457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60868)
0809 16:15:35.360497 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.360521 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:15:35.360632 20440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:33743: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
W20260809 16:15:35.406528 20451 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:15:35.406742 20451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:35.406759 20451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:35.406778 20451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:35.407883 20451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:35.407915 20451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:35.407928 20451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:35.407939 20451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:35.407961 20451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:35.409380 20451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:43159,127.16.224.189:33743,127.16.224.188:37167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab
--principal=kudu/127.16.224.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:33743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.224.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:35.409574 20451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:35.409756 20451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:35.414371 20457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:35.414466 20451 server_base.cc:1034] running on GCE node
W20260809 16:15:35.414386 20456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:35.414372 20459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:35.414844 20451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:35.415133 20451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:35.416272 20451 hybrid_clock.cc:648] HybridClock initialized: now 1786292135416255 us; error 36 us; skew 500 ppm
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:35.418596 20451 init.cc:377] Logged in from keytab as kudu/127.16.224.189@KRBTEST.COM (short username kudu)
I20260809 16:15:35.419015 20451 webserver.cc:460] Webserver started at http://127.16.224.189:34847/ using document root <none> and password file <none>
I20260809 16:15:35.419207 20451 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:35.419253 20451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:35.419377 20451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:35.420392 20451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data/instance:
uuid: "b7b8e750c9854dddab4516c0dc221f81"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "da464fd1562d360a5cbd69e6975e9f8a"
server_key_iv: "cf60d7310b792a2aa12b17b1a599fda6"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.420763 20451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/wal/instance:
uuid: "b7b8e750c9854dddab4516c0dc221f81"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "da464fd1562d360a5cbd69e6975e9f8a"
server_key_iv: "cf60d7310b792a2aa12b17b1a599fda6"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.422484 20451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 16:15:35.423125 20468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:35.423272 20451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:35.423357 20451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/wal
uuid: "b7b8e750c9854dddab4516c0dc221f81"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "da464fd1562d360a5cbd69e6975e9f8a"
server_key_iv: "cf60d7310b792a2aa12b17b1a599fda6"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.423424 20451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:35.428805 20440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } attempt: 2
I20260809 16:15:35.429194 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.429056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60870)
0809 16:15:35.429115 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.429155 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:15:35.429503 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.429436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60876)
0809 16:15:35.429468 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.429489 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:15:35.429584 20440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:33743: Network error: Client connection negotiation failed: client connection to 127.16.224.189:33743: connect: Connection refused (error 111)
I20260809 16:15:35.437440 20451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:35.439321 20451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:35.439440 20451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:35.443257 20451 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.189:33743
I20260809 16:15:35.443296 20554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.189:33743 every 8 connection(s)
I20260809 16:15:35.443611 20451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
I20260809 16:15:35.444469 20557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:35.445961 20557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:35.449460 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20451
I20260809 16:15:35.449538 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-1/wal/instance
I20260809 16:15:35.449730 17282 external_mini_cluster.cc:1428] Setting key f06c65fb7c071c20769743ccbd74b5a0
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:35.454779 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.446618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.16.224.190:43159)
0809 16:15:35.446681 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.446684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.446880 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:15:35.447335 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.447336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.447349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.447397 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.447967 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.448441 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.449369 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.449579 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.449636 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.451814 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.451816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.451816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.451824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.453050 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.453481 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.453483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.453483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.453575 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.453867 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.453869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.453869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.454033 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.454095 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.454501 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.454626 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:15:35.454849 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.446594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43159 (local address 127.0.0.1:35358)
0809 16:15:35.446767 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.446802 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.447250 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.447465 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.447468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.447704 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.447905 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.447909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.448499 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.448500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.449298 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.449302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.449647 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.449649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.449807 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.450267 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.450278 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.451718 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.453122 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.453131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.453133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.453393 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.453660 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.453661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.453661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.453812 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.454118 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.454125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.454299 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.454434 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.454613 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.16.224.188@KRBTEST.COM; defaulting to no policy
I20260809 16:15:35.458849 20557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.188@KRBTEST.COM" created.
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:15:35.467905 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.459121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60878)
0809 16:15:35.459172 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.459184 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.459228 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.461343 (+ 2115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.461345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.461497 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.461585 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.461587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.462772 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.462773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.463291 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.463293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.463734 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.463735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.463796 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.464156 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.464162 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.465369 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.466964 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.466965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.466965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.467117 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.467303 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.467304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.467304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.467351 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.467602 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.467603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.467639 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.467695 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.467801 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:15:35.467984 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.459181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.16.224.189:33743)
0809 16:15:35.461177 (+ 1996us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.461182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.461208 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:15:35.461231 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.461233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.461246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.461321 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.461667 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.462680 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.463361 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.463670 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.463729 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.465445 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.465453 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.465454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.465465 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.466890 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.467191 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.467195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.467195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.467241 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.467400 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.467401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.467401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.467543 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.467615 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.467731 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.467866 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":1961,"thread_start_us":66,"threads_started":1}
I20260809 16:15:35.468508 20557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:35.468849 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.468735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45542)
0809 16:15:35.468782 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.468827 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:15:35.468997 20478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
I20260809 16:15:35.469398 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.469311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45558)
0809 16:15:35.469334 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.469380 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260809 16:15:35.469496 20557 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:37167: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab.
Entry for principal kudu/127.16.224.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab.
I20260809 16:15:35.475080 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:37167
--webserver_interface=127.16.224.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab
--principal=kudu/127.16.224.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:43159,127.16.224.189:33743,127.16.224.188:37167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:15:35.518714 20557 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } attempt: 1
I20260809 16:15:35.519199 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.519075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45568)
0809 16:15:35.519133 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.519168 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:15:35.519606 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.519463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45572)
0809 16:15:35.519526 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.519574 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 16:15:35.519740 20557 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:37167: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
I20260809 16:15:35.520728 20440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } attempt: 3
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:15:35.527864 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.521051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60884)
0809 16:15:35.521106 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.521119 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.521181 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.521319 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.521320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.521468 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.521539 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.521541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.522464 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.522465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.523006 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.523007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.523475 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.523476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.523529 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.523960 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.523965 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.525149 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.526978 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.526979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.526980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.527145 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.527328 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.527332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.527332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.527385 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.527621 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.527622 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.527663 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.527708 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.527787 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:15:35.527899 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.521098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.16.224.189:33743)
0809 16:15:35.521159 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.521161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.521174 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:15:35.521234 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.521235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.521247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.521304 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.521608 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.522393 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.523072 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.523413 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.523481 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.525240 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.525242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.525242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.525249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.526890 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.527209 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.527210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.527211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.527267 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.527433 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.527434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.527435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.527563 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.527622 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.527740 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.527814 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:15:35.528395 20440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:35.528704 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.528616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45584)
0809 16:15:35.528659 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.528685 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:15:35.529013 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.528935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45594)
0809 16:15:35.528974 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.528998 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:15:35.529106 20440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:37167: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
I20260809 16:15:35.560233 20440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } attempt: 1
I20260809 16:15:35.560743 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.560609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45606)
0809 16:15:35.560669 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.560715 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:15:35.561082 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.561007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45612)
0809 16:15:35.561041 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.561064 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:15:35.561184 20440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:37167: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
W20260809 16:15:35.561926 20567 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:15:35.562140 20567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:35.562156 20567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:35.562175 20567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:35.563351 20567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:35.563391 20567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:35.563405 20567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:35.563422 20567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:35.563433 20567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:35.564875 20567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:43159,127.16.224.189:33743,127.16.224.188:37167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab
--principal=kudu/127.16.224.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:37167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.224.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:35.565080 20567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:35.565259 20567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:35.570014 20580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:35.570063 20572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:35.570014 20574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:35.570099 20567 server_base.cc:1034] running on GCE node
I20260809 16:15:35.570530 20567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:35.570926 20567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:35.572084 20567 hybrid_clock.cc:648] HybridClock initialized: now 1786292135572066 us; error 27 us; skew 500 ppm
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:35.574577 20567 init.cc:377] Logged in from keytab as kudu/127.16.224.188@KRBTEST.COM (short username kudu)
I20260809 16:15:35.575055 20567 webserver.cc:460] Webserver started at http://127.16.224.188:38535/ using document root <none> and password file <none>
I20260809 16:15:35.575256 20567 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:35.575299 20567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:35.575431 20567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:35.576575 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data/instance:
uuid: "297000c0977443e686c25d4450432c2a"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "39c658ea4ab885b86a9df26c214be59c"
server_key_iv: "61be52a8f1685e4c9408c53408e71b4b"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.576932 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/wal/instance:
uuid: "297000c0977443e686c25d4450432c2a"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "39c658ea4ab885b86a9df26c214be59c"
server_key_iv: "61be52a8f1685e4c9408c53408e71b4b"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.578285 20567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:15:35.579071 20594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:35.579228 20567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:35.579316 20567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/wal
uuid: "297000c0977443e686c25d4450432c2a"
format_stamp: "Formatted at 2026-08-09 16:15:35 on dist-test-slave-fgb3"
server_key: "39c658ea4ab885b86a9df26c214be59c"
server_key_iv: "61be52a8f1685e4c9408c53408e71b4b"
server_key_version: "encryptionkey@0"
I20260809 16:15:35.579386 20567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:35.587888 20557 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } attempt: 2
I20260809 16:15:35.588311 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.588192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45614)
0809 16:15:35.588249 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.588280 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:15:35.588649 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.588566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45624)
0809 16:15:35.588608 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.588631 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:15:35.588726 20557 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:37167: Network error: Client connection negotiation failed: client connection to 127.16.224.188:37167: connect: Connection refused (error 111)
I20260809 16:15:35.599002 20567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:35.601203 20567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:35.601344 20567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:35.605294 20567 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.188:37167
I20260809 16:15:35.605337 20688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.188:37167 every 8 connection(s)
I20260809 16:15:35.605679 20567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
I20260809 16:15:35.606597 20689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:35.608219 20689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:35.610837 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 20567
I20260809 16:15:35.610948 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786291868496945-17282-0/minicluster-data/master-2/wal/instance
I20260809 16:15:35.611115 17282 external_mini_cluster.cc:1428] Setting key 13ec72c06092af9240b7d8460b61cfb6
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:15:35.617576 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.608952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.16.224.190:43159)
0809 16:15:35.609015 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.609018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.609252 (+ 234us) server_negotiation.cc:407] Connection header received
0809 16:15:35.609734 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.609735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.609747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.609826 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.610471 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.611147 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.612414 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.612692 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.612746 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.614977 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.614979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.614979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.614987 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.616170 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.616550 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.616551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.616552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.616616 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.616862 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.616863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.616863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.617004 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.617053 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.617407 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.617504 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:15:35.617738 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.608928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:43159 (local address 127.0.0.1:35366)
0809 16:15:35.609093 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.609138 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.609626 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.609885 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.609889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.610138 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.610383 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.610388 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.611249 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.611250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.612324 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.612327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.612759 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.612760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.612894 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.613378 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.613388 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.614898 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.616245 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.616252 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.616254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.616484 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.616684 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.616685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.616685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.616796 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.617062 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.617068 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.617234 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.617325 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.617491 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260809 16:15:35.618389 20689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:35.619901 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.611568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.16.224.188:37167)
0809 16:15:35.611740 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.611744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.611788 (+ 44us) server_negotiation.cc:407] Connection header received
0809 16:15:35.611849 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.611852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.611872 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.611960 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.612622 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.613522 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.614236 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.614623 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.614694 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.616962 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.616967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.616967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.616973 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.618752 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.619138 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.619141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.619141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.619207 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.619378 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.619379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.619379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.619546 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.619691 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.619731 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.619802 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":111,"threads_started":1}
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:35.625142 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.618733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.16.224.189:33743)
0809 16:15:35.618783 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.618784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.618792 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:35.618828 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.618829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.618839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.618888 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.619129 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.619787 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.620597 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.620896 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.620970 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.622770 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.622772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.622772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.622780 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.624156 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.624468 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.624469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.624470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.624520 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.624695 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.624695 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.624696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.624796 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.624844 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.625004 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.625080 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:15:35.625145 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.618647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:33743 (local address 127.0.0.1:60886)
0809 16:15:35.618685 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.618695 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.618733 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.618901 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.618902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.619032 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.619069 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.619071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.619885 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.619886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.620509 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.620511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.620966 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.620967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.621010 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.621378 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.621383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.622693 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.624243 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.624244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.624244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.624397 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.624578 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.624579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.624579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.624638 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.624858 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.624858 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.624899 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.624974 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.625059 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 16:15:35.625505 20689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:35.628026 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.620683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.16.224.190:43159)
0809 16:15:35.620717 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.620719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.620741 (+ 22us) server_negotiation.cc:407] Connection header received
0809 16:15:35.620760 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.620761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.620769 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.620804 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.621320 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.621974 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.622806 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.623076 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.623137 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.625390 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.625392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.625392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.625399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.626982 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.627317 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.627318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.627318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.627364 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.627528 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.627528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.627528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.627639 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.627813 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.627861 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.627935 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:15:35.632015 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.625792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45652 (local address 127.16.224.188:37167)
0809 16:15:35.625823 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.625825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.625849 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:15:35.625867 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.625868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.625877 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.625929 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.626120 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.626802 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.627587 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.627873 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.627933 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.629627 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.629628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.629629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.629635 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.630983 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.631282 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.631283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.631284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.631348 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.631529 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.631530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.631530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.631624 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.631671 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.631819 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.631936 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":16}
I20260809 16:15:35.632067 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.625739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45652)
0809 16:15:35.625774 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.625783 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.625818 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.625932 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.625933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.626031 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.626059 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.626060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.626875 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.626876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.627516 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.627519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.627973 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.627974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.628021 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.628353 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.628358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.629561 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.631052 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.631053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.631053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.631229 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.631412 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.631413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.631413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.631474 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.631681 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.631681 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.631727 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.631802 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.631895 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:15:35.633438 20689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: Bootstrap starting.
I20260809 16:15:35.634516 20689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:35.634832 20689 log.cc:824] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:35.635711 20689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: No bootstrap required, opened a new log
I20260809 16:15:35.635792 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.628743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.16.224.189:33743)
0809 16:15:35.628779 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.628781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.628789 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:35.628808 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.628809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.628817 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.628860 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.629410 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.629861 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.630781 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.631009 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.631096 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.633454 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.633455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.633456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.633463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.634699 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.635018 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.635019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.635019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.635089 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.635290 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.635291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.635291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.635427 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.635604 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.635646 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.635715 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:35.636335 20440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } attempt: 2
I20260809 16:15:35.637257 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:35.637672 20689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.637830 20689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297000c0977443e686c25d4450432c2a, State: Initialized, Role: FOLLOWER
I20260809 16:15:35.637914 20689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.638224 20689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:35.638175 20699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.638279 20699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:35.639171 20399 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3c241757afba431fb10ddcfa600914dc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35378
W20260809 16:15:35.639923 20714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:15:35.640509 20714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:15:35.640938 20567 master_runner.cc:186] Error getting master registration for 127.16.224.190:43159: OK, instance_id { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" instance_seqno: 1786292135294844 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:15:35.643644 20567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:15:35.643796 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.636560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45668)
0809 16:15:35.636641 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.636651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.636689 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.636887 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.636888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.637042 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.637080 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.637082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.637863 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.637864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.638864 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.638867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.639344 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.639345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.639405 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.639800 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.639805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.641022 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.642730 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.642731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.642732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.642934 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.643136 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.643137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.643137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.643195 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.643501 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.643502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.643548 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.643608 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.643702 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 16:15:35.644068 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.636604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.16.224.188:37167)
0809 16:15:35.636802 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.636804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.636811 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:35.636828 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.636829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.636838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.636883 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.637150 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.637776 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.638948 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.639219 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.639288 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.641113 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.641114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.641114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.641120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.642641 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.643014 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.643015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.643016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.643071 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.643255 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.643256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.643256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.643397 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.643471 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.643916 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.643987 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260809 16:15:35.645243 20440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc: Bootstrap starting.
I20260809 16:15:35.645818 20440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:35.646066 20440 log.cc:824] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:35.646872 20440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc: No bootstrap required, opened a new log
I20260809 16:15:35.647809 20440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.647921 20440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c241757afba431fb10ddcfa600914dc, State: Initialized, Role: FOLLOWER
I20260809 16:15:35.647994 20440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.648223 20719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.648279 20440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:35.648339 20719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:35.648365 20399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:15:35.648476 20399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:35.649216 20399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.650063 20650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c241757afba431fb10ddcfa600914dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "297000c0977443e686c25d4450432c2a"
I20260809 16:15:35.650179 20650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:35.651039 20650 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c241757afba431fb10ddcfa600914dc in term 1.
I20260809 16:15:35.651250 20345 leader_election.cc:302] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297000c0977443e686c25d4450432c2a, 3c241757afba431fb10ddcfa600914dc; no voters:
I20260809 16:15:35.651376 20719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:35.651526 20719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [term 1 LEADER]: Becoming Leader. State: Replica: 3c241757afba431fb10ddcfa600914dc, State: Running, Role: LEADER
I20260809 16:15:35.651613 20719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.652117 20399 leader_election.cc:288] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [CANDIDATE]: Term 1 election: Requested vote from peers b7b8e750c9854dddab4516c0dc221f81 (127.16.224.189:33743), 297000c0977443e686c25d4450432c2a (127.16.224.188:37167)
I20260809 16:15:35.653169 20315 master_runner.cc:186] Error getting master registration for 127.16.224.189:33743: OK, instance_id { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" instance_seqno: 1786292135442137 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:15:35.655755 20315 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:15:35.655993 20746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c241757afba431fb10ddcfa600914dc. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.656082 20746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:35.656265 20764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:35.656729 20764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:35.661999 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.653193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.16.224.189:33743)
0809 16:15:35.653230 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.653232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.653417 (+ 185us) server_negotiation.cc:407] Connection header received
0809 16:15:35.653519 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.653520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.653532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.653562 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.654256 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.654999 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.656380 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.656705 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.656742 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.658924 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.658926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.658926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.658944 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.660509 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.661121 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.661122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.661122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.661174 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.661348 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.661349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.661349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.661491 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.661594 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.661838 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.661909 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:15:35.662819 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.653164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.16.224.190:43159)
0809 16:15:35.653202 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.653204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.654370 (+ 1166us) server_negotiation.cc:407] Connection header received
0809 16:15:35.654457 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.654458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.654467 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.654495 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.655126 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.655807 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.656938 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.657194 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.657232 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.658661 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.658662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.658663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.658669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.660544 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.661053 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.661055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.661055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.661113 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.661477 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.661479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.661479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.661599 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.661660 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.662642 (+ 982us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.662729 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:35.663280 20650 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [term 1 FOLLOWER]: Refusing update from remote peer 3c241757afba431fb10ddcfa600914dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:35.663601 20746 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:35.664959 20699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c241757afba431fb10ddcfa600914dc. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.665061 20699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:35.666237 20746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.666357 20746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:35.666859 20766 mvcc.cc:205] Tried to move back new op lower bound from 7316652587674075136 to 7316652587629645824. Current Snapshot: MvccSnapshot[applied={T|T < 7316652587674075136}]
I20260809 16:15:35.667162 20699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.667254 20699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:35.670159 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.653481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.16.224.188:37167)
0809 16:15:35.653512 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.653513 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.661990 (+ 8477us) server_negotiation.cc:407] Connection header received
0809 16:15:35.662057 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.662058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.662071 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.662113 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.663941 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.664582 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.665476 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.665733 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.665777 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.667149 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.667151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.667151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.667157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.668833 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.669308 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.669309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.669310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.669371 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.669602 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.669603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.669603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.669741 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.669822 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.670008 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.670077 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:15:35.670833 20764 catalog_manager.cc:1347] Generated new cluster ID: 207df89524ee47a583d70b3b5e5a0c50
I20260809 16:15:35.670886 20764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:35.679845 20557 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } attempt: 3
Aug 09 16:15:35 dist-test-slave-fgb3 krb5kdc[20287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:15:35.688474 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.680085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:37167 (local address 127.0.0.1:45696)
0809 16:15:35.680131 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:35.680142 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:35.680177 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:35.680346 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:35.680346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:35.680496 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:35.680538 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.680540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.681400 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.681401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.682199 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:35.682200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.682857 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.682858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:35.682924 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.683373 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:35.683377 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:35.685716 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:35.687357 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.687358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.687358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.687555 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.687808 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:35.687809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:35.687810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:35.687861 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:35.688189 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:35.688190 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:35.688243 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:15:35.688284 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:15:35.688380 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:15:35.689208 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.680126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.16.224.188:37167)
0809 16:15:35.680178 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.680180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.680187 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:35.680237 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.680238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.680247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.680300 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.680614 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.681316 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.682418 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.682747 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.682796 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.685801 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:35.685802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:35.685803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:35.685810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:35.687254 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.687627 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.687628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.687628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.687706 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:35.687923 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:35.687923 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:35.687924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:35.688118 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:35.688885 (+ 767us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.688950 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.689070 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:15:35.690847 20557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81: Bootstrap starting.
I20260809 16:15:35.691529 20557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:35.691778 20557 log.cc:824] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:35.693332 20557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81: No bootstrap required, opened a new log
I20260809 16:15:35.695139 20557 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.695255 20557 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b8e750c9854dddab4516c0dc221f81, State: Initialized, Role: FOLLOWER
I20260809 16:15:35.695327 20557 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } }
I20260809 16:15:35.695606 20557 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:35.696816 20513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:35.698264 20513 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [term 1 FOLLOWER]: Refusing update from remote peer 3c241757afba431fb10ddcfa600914dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:35.696880 20514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c241757afba431fb10ddcfa600914dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b7b8e750c9854dddab4516c0dc221f81"
I20260809 16:15:35.698467 20764 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:15:35.698761 20746 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:35.699121 20764 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:15:35.699144 20514 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c241757afba431fb10ddcfa600914dc in term 1.
I20260809 16:15:35.700766 20775 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.700881 20775 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:35.700937 20775 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c241757afba431fb10ddcfa600914dc. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.700982 20775 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:35.701011 20775 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.701048 20775 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:35.702502 20795 mvcc.cc:205] Tried to move back new op lower bound from 7316652587816579072 to 7316652587629645824. Current Snapshot: MvccSnapshot[applied={T|T < 7316652587674075136}]
I20260809 16:15:35.706590 20719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c241757afba431fb10ddcfa600914dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c241757afba431fb10ddcfa600914dc" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 43159 } } peers { permanent_uuid: "b7b8e750c9854dddab4516c0dc221f81" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 33743 } } peers { permanent_uuid: "297000c0977443e686c25d4450432c2a" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 37167 } } }
I20260809 16:15:35.706691 20719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:35.707636 20764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c241757afba431fb10ddcfa600914dc: Generated new TSK 0
I20260809 16:15:35.707861 20764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:35.707886 20792 catalog_manager.cc:1259] Loaded cluster ID: 207df89524ee47a583d70b3b5e5a0c50
I20260809 16:15:35.707950 20792 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81: loading cluster ID for follower catalog manager: success
I20260809 16:15:35.709055 20792 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81: acquiring CA information for follower catalog manager: success
I20260809 16:15:35.709371 20792 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b7b8e750c9854dddab4516c0dc221f81: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 16:15:35.734503 20442 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.732834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.16.224.190:43159)
0809 16:15:35.732878 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.732880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.733828 (+ 948us) server_negotiation.cc:407] Connection header received
0809 16:15:35.733927 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.733929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.733940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.733968 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.734433 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35392: BlockingRecv error: recv got EOF from 127.0.0.1:35392 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:15:35.735385 20562 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.733244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.16.224.189:33743)
0809 16:15:35.733284 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.733286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.734711 (+ 1425us) server_negotiation.cc:407] Connection header received
0809 16:15:35.734867 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.734868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.734879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.734907 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.735327 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60922: BlockingRecv error: recv got EOF from 127.0.0.1:60922 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:15:35.735989 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.733539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.16.224.188:37167)
0809 16:15:35.733576 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.733578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.734994 (+ 1416us) server_negotiation.cc:407] Connection header received
0809 16:15:35.735600 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.735602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.735611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.735638 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.735936 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45702: BlockingRecv error: recv got EOF from 127.0.0.1:45702 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:35.746143 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.742612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.16.224.189:33743)
0809 16:15:35.742659 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.742661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.742893 (+ 232us) server_negotiation.cc:407] Connection header received
0809 16:15:35.743007 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.743008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.743054 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.743095 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.743308 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.744051 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.745175 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.745469 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.745516 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.745750 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.745876 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.745930 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.746049 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.746070 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:15:35.749576 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.748671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.16.224.188:37167)
0809 16:15:35.748711 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.748713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.749002 (+ 289us) server_negotiation.cc:407] Connection header received
0809 16:15:35.749111 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.749113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.749131 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.749161 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.749526 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45718: BlockingRecv error: recv got EOF from 127.0.0.1:45718 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:35.755026 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.751802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.16.224.189:33743)
0809 16:15:35.751838 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.751840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.752086 (+ 246us) server_negotiation.cc:407] Connection header received
0809 16:15:35.752180 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.752181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.752190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.752217 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.752392 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.753074 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.754137 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.754518 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.754564 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.754774 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.754810 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.754837 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.754937 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.754957 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:15:35.757380 20688 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.16.224.188:37167: Invalid argument: socket address is ambiguous
W20260809 16:15:35.758303 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.757508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.16.224.188:37167)
0809 16:15:35.757543 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.757544 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.757761 (+ 217us) server_negotiation.cc:407] Connection header received
0809 16:15:35.757856 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.757858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.757868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.757898 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.758257 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45726: BlockingRecv error: recv got EOF from 127.0.0.1:45726 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:35.764598 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.761368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.16.224.189:33743)
0809 16:15:35.761402 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.761403 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.761657 (+ 254us) server_negotiation.cc:407] Connection header received
0809 16:15:35.761748 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.761749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.761759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.761784 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.761942 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.762756 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.763814 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.764093 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.764137 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.764334 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.764367 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.764393 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.764512 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.764531 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:15:35.767554 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.766777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.16.224.188:37167)
0809 16:15:35.766814 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.766815 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.767026 (+ 211us) server_negotiation.cc:407] Connection header received
0809 16:15:35.767117 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.767118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.767129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.767157 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.767506 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45734: BlockingRecv error: recv got EOF from 127.0.0.1:45734 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:35.776010 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.772672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60962 (local address 127.16.224.189:33743)
0809 16:15:35.772712 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.772714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.772981 (+ 267us) server_negotiation.cc:407] Connection header received
0809 16:15:35.773085 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.773086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.773096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.773124 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.773303 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.774055 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.775234 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.775522 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.775565 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.775760 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.775794 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.775820 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.775914 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.775932 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:15:35.779261 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.778431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.16.224.188:37167)
0809 16:15:35.778478 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.778480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.778690 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:15:35.778781 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.778782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.778793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.778821 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.779198 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45738: BlockingRecv error: recv got EOF from 127.0.0.1:45738 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:15:35.791913 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.788540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.16.224.189:33743)
0809 16:15:35.788586 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.788588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.788893 (+ 305us) server_negotiation.cc:407] Connection header received
0809 16:15:35.788998 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.789000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.789010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.789038 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.789214 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.789925 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.791085 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.791363 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.791411 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.791641 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.791679 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.791706 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.791810 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.791830 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:15:35.795475 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.794674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.16.224.188:37167)
0809 16:15:35.794716 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.794718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.794923 (+ 205us) server_negotiation.cc:407] Connection header received
0809 16:15:35.795009 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.795011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.795021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.795047 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.795403 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45746: BlockingRecv error: recv got EOF from 127.0.0.1:45746 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:15:35.815990 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.812686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.16.224.189:33743)
0809 16:15:35.812729 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.812731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.813000 (+ 269us) server_negotiation.cc:407] Connection header received
0809 16:15:35.813098 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.813100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.813110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.813135 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.813299 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.814054 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.815178 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.815481 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.815519 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.815731 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.815768 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.815794 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.815889 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.815909 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:15:35.818956 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.818157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.16.224.188:37167)
0809 16:15:35.818192 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.818194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.818428 (+ 234us) server_negotiation.cc:407] Connection header received
0809 16:15:35.818523 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.818524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.818536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.818563 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.818902 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45756: BlockingRecv error: recv got EOF from 127.0.0.1:45756 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:15:35.854619 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.852346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.16.224.189:33743)
0809 16:15:35.852392 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.852394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.852544 (+ 150us) server_negotiation.cc:407] Connection header received
0809 16:15:35.852594 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.852596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.852607 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.852651 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.852771 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.853249 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.853999 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.854183 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.854237 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.854398 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.854428 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.854466 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.854519 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.854539 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:15:35.856962 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.856232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.16.224.188:37167)
0809 16:15:35.856280 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.856282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.856533 (+ 251us) server_negotiation.cc:407] Connection header received
0809 16:15:35.856566 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.856567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.856579 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.856627 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.856912 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45770: BlockingRecv error: recv got EOF from 127.0.0.1:45770 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:15:35.924561 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:35.922367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.16.224.189:33743)
0809 16:15:35.922443 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.922446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.922535 (+ 89us) server_negotiation.cc:407] Connection header received
0809 16:15:35.922596 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.922597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.922606 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.922655 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:35.922757 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.923234 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.924026 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:35.924208 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:35.924256 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:35.924369 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:35.924396 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:35.924435 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:35.924469 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 16:15:35.924483 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 16:15:35.926965 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:35.926277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.16.224.188:37167)
0809 16:15:35.926348 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:35.926350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:35.926415 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:15:35.926504 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:35.926505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:35.926518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:35.926563 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:35.926909 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45780: BlockingRecv error: recv got EOF from 127.0.0.1:45780 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:36.057974 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:36.055935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.16.224.189:33743)
0809 16:15:36.056035 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.056037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.056082 (+ 45us) server_negotiation.cc:407] Connection header received
0809 16:15:36.056137 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.056138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.056148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.056200 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:36.056302 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.056787 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.057462 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.057638 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.057680 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:36.057777 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:36.057804 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:36.057834 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:36.057879 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 16:15:36.057894 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 16:15:36.060099 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:36.059511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.16.224.188:37167)
0809 16:15:36.059577 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.059579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.059639 (+ 60us) server_negotiation.cc:407] Connection header received
0809 16:15:36.059706 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.059707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.059717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.059761 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:36.060044 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45790: BlockingRecv error: recv got EOF from 127.0.0.1:45790 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:15:36.319242 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:36.317189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.16.224.189:33743)
0809 16:15:36.317258 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.317260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.317285 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:15:36.317317 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.317318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.317328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.317377 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:36.317506 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.317981 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.318716 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.318908 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.318956 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:36.319053 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:36.319080 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:36.319111 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:36.319149 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:15:36.319164 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 16:15:36.321360 20696 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:36.320848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.16.224.188:37167)
0809 16:15:36.320912 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.320914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.320930 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:15:36.320954 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.320955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.320967 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.321010 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:36.321309 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45798: BlockingRecv error: recv got EOF from 127.0.0.1:45798 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 16:15:36.606688 20682 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.16.224.188:37167
I20260809 16:15:36.641044 20714 catalog_manager.cc:1259] Loaded cluster ID: 207df89524ee47a583d70b3b5e5a0c50
I20260809 16:15:36.641121 20714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: loading cluster ID for follower catalog manager: success
I20260809 16:15:36.641938 20714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: acquiring CA information for follower catalog manager: success
I20260809 16:15:36.642248 20714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 297000c0977443e686c25d4450432c2a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:15:36.836655 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:36.834397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.16.224.189:33743)
0809 16:15:36.834512 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.834515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.834546 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:15:36.834632 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.834633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.834645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.834695 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:36.834818 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.835310 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.836046 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.836240 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.836294 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:36.836426 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:36.836457 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:36.836498 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:36.836546 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:15:36.836568 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260809 16:15:36.840669 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:36.838088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.16.224.188:37167)
0809 16:15:36.838210 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.838212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.838220 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:36.838287 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.838288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.838388 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.838465 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:36.838564 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.839120 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.839925 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.840129 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.840181 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:36.840320 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:36.840436 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:36.840504 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:36.840527 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 16:15:36.840547 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260809 16:15:36.845363 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:36.842345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.16.224.190:43159)
0809 16:15:36.842483 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.842486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.842586 (+ 100us) server_negotiation.cc:407] Connection header received
0809 16:15:36.842642 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.842644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.842700 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.842781 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:36.842864 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.843442 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.844416 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.844701 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.844747 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:36.844940 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:36.845089 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:36.845142 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:36.845239 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 16:15:36.845272 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260809 16:15:36.845371 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:36.842228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32784 (local address 127.16.224.189:33743)
0809 16:15:36.842285 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.842286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.842541 (+ 255us) server_negotiation.cc:407] Connection header received
0809 16:15:36.842599 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.842600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.842611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.842672 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:36.842794 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.843504 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.844271 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.844520 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.844579 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:36.845133 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:36.845164 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:36.845214 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:36.845289 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:15:36.845313 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:15:36.845410 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:36.842387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.16.224.188:37167)
0809 16:15:36.842430 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:36.842432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:36.842666 (+ 234us) server_negotiation.cc:407] Connection header received
0809 16:15:36.842746 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:36.842747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:36.842755 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:36.842798 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:36.842994 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.843668 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.844592 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:36.844819 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:36.844887 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:36.845037 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:36.845071 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:36.845119 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:36.845314 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 16:15:36.845339 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:15:36.847113 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20315
I20260809 16:15:36.859793 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20451
I20260809 16:15:36.868816 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 20567
2026-08-09T16:15:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2950 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:15:36 dist-test-slave-fgb3 krb5kdc[21027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:15:36 dist-test-slave-fgb3 krb5kdc[21027](info): set up 2 sockets
Aug 09 16:15:36 dist-test-slave-fgb3 krb5kdc[21027](info): commencing operation
krb5kdc: starting...
W20260809 16:15:38.009689 17282 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/test-user.keytab.
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:15:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:15:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.224.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
Entry for principal kudu/127.16.224.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab.
I20260809 16:15:38.124257 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:39995
--webserver_interface=127.16.224.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:39995,127.16.224.189:39559,127.16.224.188:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:15:38.205091 21047 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:15:38.205309 21047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:38.205334 21047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:38.205359 21047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:38.206506 21047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:38.206544 21047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:38.206562 21047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:38.206579 21047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:38.206593 21047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:38.208086 21047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:39995,127.16.224.189:39559,127.16.224.188:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.190.keytab
--principal=kudu/127.16.224.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.190:39995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.224.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:38.208344 21047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:38.208554 21047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:38.213254 21053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:38.213266 21058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:38.213447 21047 server_base.cc:1034] running on GCE node
W20260809 16:15:38.213266 21061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:38.213759 21047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:38.214038 21047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:38.215178 21047 hybrid_clock.cc:648] HybridClock initialized: now 1786292138215159 us; error 36 us; skew 500 ppm
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:38.222841 21047 init.cc:377] Logged in from keytab as kudu/127.16.224.190@KRBTEST.COM (short username kudu)
I20260809 16:15:38.223291 21047 webserver.cc:460] Webserver started at http://127.16.224.190:40873/ using document root <none> and password file <none>
I20260809 16:15:38.223518 21047 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:38.223569 21047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:38.223701 21047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:38.224802 21047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data/instance:
uuid: "515360f971ad408c93e8cb0f4efd3faf"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "e9e09c7ceb07638297cd57329a49e879"
server_key_iv: "f54043d2a32e92d78ef90b9ccbdc997e"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.225150 21047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/wal/instance:
uuid: "515360f971ad408c93e8cb0f4efd3faf"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "e9e09c7ceb07638297cd57329a49e879"
server_key_iv: "f54043d2a32e92d78ef90b9ccbdc997e"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.226446 21047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:15:38.227144 21074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:38.227293 21047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:15:38.227388 21047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/wal
uuid: "515360f971ad408c93e8cb0f4efd3faf"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "e9e09c7ceb07638297cd57329a49e879"
server_key_iv: "f54043d2a32e92d78ef90b9ccbdc997e"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.227456 21047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:38.237077 21047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:38.238829 21047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:38.238963 21047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:38.242786 21047 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.190:39995
I20260809 16:15:38.242820 21165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.190:39995 every 8 connection(s)
I20260809 16:15:38.243140 21047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/data/info.pb
I20260809 16:15:38.244020 21166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:38.245591 21166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:38.248962 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 21047
I20260809 16:15:38.249034 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-0/wal/instance
I20260809 16:15:38.249203 17282 external_mini_cluster.cc:1428] Setting key c3cab656c12d49a8bde77d18b063c253
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:38.254387 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.246230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.16.224.190:39995)
0809 16:15:38.246392 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.246400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.246507 (+ 107us) server_negotiation.cc:407] Connection header received
0809 16:15:38.247014 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.247016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.247032 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.247112 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.247579 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.248374 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.249317 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.249601 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.249697 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.251645 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.251652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.251652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.251658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.253018 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.253347 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.253350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.253350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.253420 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.253603 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.253604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.253604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.253793 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.253860 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.254041 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.254223 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260809 16:15:38.254390 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.246200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39995 (local address 127.0.0.1:59130)
0809 16:15:38.246385 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.246423 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.246897 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.247124 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.247126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.247355 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.247513 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.247518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.248473 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.248474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.249225 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.249228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.249675 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.249676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.249742 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.250194 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.250202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.251567 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.253094 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.253100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.253102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.253278 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.253475 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.253476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.253476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.253550 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.253852 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.253857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.253916 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.253998 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.254151 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260809 16:15:38.255024 21166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:38.255406 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.255303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49010)
0809 16:15:38.255345 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.255387 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:15:38.255517 21078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
I20260809 16:15:38.255861 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.255781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49026)
0809 16:15:38.255817 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.255840 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:15:38.255978 21166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:39559: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.16.224.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab.
Entry for principal kudu/127.16.224.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab.
I20260809 16:15:38.273123 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:39559
--webserver_interface=127.16.224.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab
--principal=kudu/127.16.224.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:39995,127.16.224.189:39559,127.16.224.188:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:15:38.305228 21166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } attempt: 1
I20260809 16:15:38.305801 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.305651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49034)
0809 16:15:38.305715 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.305766 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:15:38.306144 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.306051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49036)
0809 16:15:38.306093 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.306121 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:15:38.306253 21166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:39559: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
W20260809 16:15:38.355433 21179 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:15:38.355638 21179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:38.355654 21179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:38.355675 21179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:38.356812 21179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:38.356850 21179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:38.356863 21179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:38.356890 21179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:38.356905 21179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:38.358464 21179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:39995,127.16.224.189:39559,127.16.224.188:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.189.keytab
--principal=kudu/127.16.224.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.189:39559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.224.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:38.358697 21179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:38.358911 21179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:38.363468 21185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:38.363512 21186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:38.363539 21179 server_base.cc:1034] running on GCE node
W20260809 16:15:38.363469 21189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:38.363891 21179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:38.364142 21179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:38.365276 21179 hybrid_clock.cc:648] HybridClock initialized: now 1786292138365261 us; error 34 us; skew 500 ppm
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:38.367728 21179 init.cc:377] Logged in from keytab as kudu/127.16.224.189@KRBTEST.COM (short username kudu)
I20260809 16:15:38.368146 21179 webserver.cc:460] Webserver started at http://127.16.224.189:33485/ using document root <none> and password file <none>
I20260809 16:15:38.368350 21179 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:38.368396 21179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:38.368520 21179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:38.369565 21179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data/instance:
uuid: "d00958ee0db642ffa117710072d62254"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "22366f592849d54708a6d9b62b4626d0"
server_key_iv: "8d87c5976e898858b0f8e4a77525b0cd"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.369891 21179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/wal/instance:
uuid: "d00958ee0db642ffa117710072d62254"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "22366f592849d54708a6d9b62b4626d0"
server_key_iv: "8d87c5976e898858b0f8e4a77525b0cd"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.371186 21179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:38.371973 21198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:38.372161 21179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:15:38.372260 21179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/wal
uuid: "d00958ee0db642ffa117710072d62254"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "22366f592849d54708a6d9b62b4626d0"
server_key_iv: "8d87c5976e898858b0f8e4a77525b0cd"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.372326 21179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:38.374372 21166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } attempt: 2
I20260809 16:15:38.374845 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.374649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49050)
0809 16:15:38.374756 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.374804 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 16:15:38.375129 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.375070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49062)
0809 16:15:38.375093 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.375115 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260809 16:15:38.375216 21166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.189:39559: Network error: Client connection negotiation failed: client connection to 127.16.224.189:39559: connect: Connection refused (error 111)
I20260809 16:15:38.398763 21179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:38.400988 21179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:38.401124 21179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:38.405059 21179 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.189:39559
I20260809 16:15:38.405112 21282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.189:39559 every 8 connection(s)
I20260809 16:15:38.405449 21179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/data/info.pb
I20260809 16:15:38.406407 21284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:38.407982 21284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:38.408597 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 21179
I20260809 16:15:38.408692 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-1/wal/instance
I20260809 16:15:38.408910 17282 external_mini_cluster.cc:1428] Setting key 081c45730263ff6d228cf39c016c0cfa
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.224.188@KRBTEST.COM; defaulting to no policy
I20260809 16:15:38.416874 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.408666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59146 (local address 127.16.224.190:39995)
0809 16:15:38.408765 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.408767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.408998 (+ 231us) server_negotiation.cc:407] Connection header received
0809 16:15:38.409583 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.409584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.409594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.409635 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.410398 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.410885 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.411872 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.412096 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.412161 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.414403 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.414405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.414405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.414413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.415635 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.416001 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.416002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.416002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.416054 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.416265 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.416266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.416266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.416376 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.416431 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.416687 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.416783 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 16:15:38.417014 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.408601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39995 (local address 127.0.0.1:59146)
0809 16:15:38.408851 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.408899 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.409483 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.409708 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.409712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.410025 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.410289 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.410295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.410978 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.410980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.411764 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.411767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.412184 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.412186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.412333 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.412810 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.412819 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.414301 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.415713 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.415720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.415723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.415933 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.416107 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.416108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.416108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.416209 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.416430 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.416437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.416570 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.416657 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.416804 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":73,"threads_started":1}
I20260809 16:15:38.417531 21284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.224.188@KRBTEST.COM" created.
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:15:38.424721 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.417772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49068)
0809 16:15:38.417813 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.417823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.417858 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.418213 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.418214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.418416 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.418468 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.418469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.419607 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.419608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.420236 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.420239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.420672 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.420673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.420736 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.421062 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.421066 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.422293 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.423822 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.423823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.423823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.423971 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.424137 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.424137 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.424138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.424189 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.424422 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.424422 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.424467 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.424529 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.424619 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:15:38.424790 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.417862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49068 (local address 127.16.224.189:39559)
0809 16:15:38.418043 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.418047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.418070 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:15:38.418089 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.418091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.418102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.418176 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.418537 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.419535 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.420311 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.420607 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.420658 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.422383 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.422391 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.422391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.422397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.423757 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.424035 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.424038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.424038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.424085 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.424237 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.424238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.424238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.424364 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.424432 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.424563 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.424670 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260809 16:15:38.425418 21284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:38.425758 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.425630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42724)
0809 16:15:38.425676 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.425730 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:15:38.425889 21203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
I20260809 16:15:38.426273 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.426169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42732)
0809 16:15:38.426217 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.426248 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:15:38.426397 21284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:38965: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.224.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab.
Entry for principal kudu/127.16.224.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab.
I20260809 16:15:38.432885 17282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLoTDwK/build/release/bin/kudu
/tmp/dist-test-taskLoTDwK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:38965
--webserver_interface=127.16.224.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab
--principal=kudu/127.16.224.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.224.148:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.224.190:39995,127.16.224.189:39559,127.16.224.188:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:15:38.466388 21166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } attempt: 3
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:15:38.473923 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.466897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.16.224.189:39559)
0809 16:15:38.466960 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.466963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.466975 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:15:38.467008 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.467010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.467021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.467092 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.467421 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.468180 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.468967 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.469303 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.469392 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.471533 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.471535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.471535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.471544 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.472843 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.473235 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.473237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.473237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.473299 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.473482 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.473483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.473483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.473590 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.473638 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.473775 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.473853 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:15:38.473951 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.466808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49082)
0809 16:15:38.466872 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.466887 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.466941 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.467114 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.467115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.467277 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.467350 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.467352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.468297 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.468299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.468869 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.468871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.469415 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.469417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.469504 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.470036 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.470042 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.471436 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.472921 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.472923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.472923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.473143 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.473363 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.473364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.473364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.473431 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.473640 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.473640 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.473691 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.473750 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.473853 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:15:38.474591 21166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:38.474970 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.474826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42748)
0809 16:15:38.474870 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.474909 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:15:38.475353 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.475256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42750)
0809 16:15:38.475297 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.475332 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:15:38.475445 21166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:38965: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
I20260809 16:15:38.475590 21284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } attempt: 1
I20260809 16:15:38.475898 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.475795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42760)
0809 16:15:38.475841 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.475872 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:15:38.476208 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.476135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42776)
0809 16:15:38.476170 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.476193 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:15:38.476289 21284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:38965: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
I20260809 16:15:38.506594 21166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } attempt: 1
I20260809 16:15:38.507079 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.506915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42792)
0809 16:15:38.506985 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.507037 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:15:38.507443 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.507339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42806)
0809 16:15:38.507387 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.507419 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:15:38.507543 21166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:38965: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
W20260809 16:15:38.518496 21304 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:15:38.518718 21304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:15:38.518735 21304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:15:38.518756 21304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:15:38.519925 21304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:15:38.519965 21304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:15:38.519979 21304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:15:38.520001 21304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:15:38.520022 21304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:15:38.521514 21304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.224.148:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.224.190:39995,127.16.224.189:39559,127.16.224.188:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/krb5kdc/kudu_127.16.224.188.keytab
--principal=kudu/127.16.224.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.224.188:38965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.224.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:15:38.521761 21304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:15:38.521987 21304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:15:38.526636 21316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:38.526655 21312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:15:38.526621 21311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:15:38.526943 21304 server_base.cc:1034] running on GCE node
I20260809 16:15:38.527191 21304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:15:38.527467 21304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:15:38.528618 21304 hybrid_clock.cc:648] HybridClock initialized: now 1786292138528604 us; error 29 us; skew 500 ppm
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:15:38.531150 21304 init.cc:377] Logged in from keytab as kudu/127.16.224.188@KRBTEST.COM (short username kudu)
I20260809 16:15:38.531579 21304 webserver.cc:460] Webserver started at http://127.16.224.188:37475/ using document root <none> and password file <none>
I20260809 16:15:38.531807 21304 fs_manager.cc:359] Metadata directory not provided
I20260809 16:15:38.531865 21304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:15:38.531988 21304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:15:38.533030 21304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data/instance:
uuid: "350b00408fe9450c8a6679a0df360aba"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "f105e11b36969dfd7eb4acd4039ca99a"
server_key_iv: "8c0e88f1592a7b0dff8c6275f4b2e602"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.533367 21304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/wal/instance:
uuid: "350b00408fe9450c8a6679a0df360aba"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "f105e11b36969dfd7eb4acd4039ca99a"
server_key_iv: "8c0e88f1592a7b0dff8c6275f4b2e602"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.534601 21304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:15:38.535250 21327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:15:38.535395 21304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:15:38.535480 21304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data,/tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/wal
uuid: "350b00408fe9450c8a6679a0df360aba"
format_stamp: "Formatted at 2026-08-09 16:15:38 on dist-test-slave-fgb3"
server_key: "f105e11b36969dfd7eb4acd4039ca99a"
server_key_iv: "8c0e88f1592a7b0dff8c6275f4b2e602"
server_key_version: "encryptionkey@0"
I20260809 16:15:38.535537 21304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:15:38.544425 21284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } attempt: 2
I20260809 16:15:38.544854 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.544721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42814)
0809 16:15:38.544779 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.544821 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:15:38.545143 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.545078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42816)
0809 16:15:38.545107 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.545128 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 16:15:38.545233 21284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.224.188:38965: Network error: Client connection negotiation failed: client connection to 127.16.224.188:38965: connect: Connection refused (error 111)
I20260809 16:15:38.549064 21304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:15:38.551126 21304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:15:38.551235 21304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:15:38.555096 21304 rpc_server.cc:307] RPC server started. Bound to: 127.16.224.188:38965
I20260809 16:15:38.555130 21411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.224.188:38965 every 8 connection(s)
I20260809 16:15:38.555446 21304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/data/info.pb
I20260809 16:15:38.556279 21412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:15:38.557576 17282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLoTDwK/build/release/bin/kudu as pid 21304
I20260809 16:15:38.557654 17282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLoTDwK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786291868496945-17282-0/minicluster-data/master-2/wal/instance
I20260809 16:15:38.557838 17282 external_mini_cluster.cc:1428] Setting key db2fcb311cbcb7d7549e86fe29b683b0
I20260809 16:15:38.557822 21412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:15:38.566542 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.558540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59154 (local address 127.16.224.190:39995)
0809 16:15:38.558598 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.558600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.558735 (+ 135us) server_negotiation.cc:407] Connection header received
0809 16:15:38.559156 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.559157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.559169 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.559220 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.559781 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.560275 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.561180 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.561376 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.561432 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.563581 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.563583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.563583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.563590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.565123 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.565572 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.565573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.565574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.565632 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.565882 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.565883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.565883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.566019 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.566080 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.566372 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.566451 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:15:38.566936 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.558362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.16.224.188:38965)
0809 16:15:38.558543 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.558553 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.558590 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:15:38.559031 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.559039 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.559069 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.559160 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.559783 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.560668 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.561395 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.561686 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.561742 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.564017 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.564024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.564025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.564032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.565581 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.565940 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.565943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.565943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.565994 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.566174 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.566175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.566175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.566296 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.566498 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.566582 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.566652 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260809 16:15:38.566938 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.558466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.190:39995 (local address 127.0.0.1:59154)
0809 16:15:38.558615 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.558659 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.559048 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.559256 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.559259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.559528 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.559695 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.559699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.560355 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.560356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.561094 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.561096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.561446 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.561447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.561565 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.561996 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.562005 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.563499 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.565213 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.565219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.565221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.565506 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.565687 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.565687 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.565688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.565820 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.566076 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.566083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.566231 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.566340 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.566493 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260809 16:15:38.567518 21412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.190@KRBTEST.COM
I20260809 16:15:38.573559 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.567769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.189:39559 (local address 127.0.0.1:49090)
0809 16:15:38.567813 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.567821 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.567852 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.568001 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.568001 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.568102 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.568137 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.568138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.568722 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.568723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.569219 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.569221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.569595 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.569595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.569652 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.569978 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.569983 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.571114 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.572669 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.572670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.572670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.572817 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.572999 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.572999 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.572999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.573041 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.573266 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.573267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.573312 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.573386 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.573479 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:15:38.573570 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.567845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.16.224.189:39559)
0809 16:15:38.567889 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.567890 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.567898 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:38.567921 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.567922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.567930 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.567998 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.568197 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.568660 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.569284 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.569537 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.569583 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.571196 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.571198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.571198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.571206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.572598 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.572883 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.572884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.572885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.572942 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.573076 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.573076 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.573077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.573197 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.573251 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.573408 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.573481 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:15:38.574060 21412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:15:38.574142 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.567659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.16.224.190:39995)
0809 16:15:38.567696 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.567697 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.567703 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:15:38.567720 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.567721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.567728 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.567771 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.568266 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.568770 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.569389 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.569555 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.569610 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.571718 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.571719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.571719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.571727 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.573120 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.573417 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.573418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.573419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.573473 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.573671 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.573672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.573672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.573768 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.573944 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.573990 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.574058 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.188@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.224.189@KRBTEST.COM
I20260809 16:15:38.580611 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.574448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42838)
0809 16:15:38.574504 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.574514 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.574558 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.574673 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.574674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.574808 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.574844 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.574846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.575646 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.575647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.576333 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.576335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.576754 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.576754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.576797 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.577101 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.577106 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.578391 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.579765 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.579766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.579767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.579922 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.580093 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.580094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.580094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.580140 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.580333 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.580334 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.580368 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.580426 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.580518 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:15:38.580669 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.574469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42838 (local address 127.16.224.188:38965)
0809 16:15:38.574503 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.574504 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.574545 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:15:38.574610 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.574611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.574620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.574671 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.574907 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.575580 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.576392 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.576688 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.576757 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.578465 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.578467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.578467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.578474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.579688 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.579989 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.579989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.579990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.580041 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.580187 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.580188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.580188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.580283 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.580335 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.580459 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.580567 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":17}
I20260809 16:15:38.582226 21412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: Bootstrap starting.
I20260809 16:15:38.582685 21166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } attempt: 2
I20260809 16:15:38.582937 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.574794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.16.224.189:39559)
0809 16:15:38.574837 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.574838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.574846 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:38.574866 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.574867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.574875 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.574922 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.575423 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.576006 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.576772 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.577027 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.577088 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.579445 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.579446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.579446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.579452 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.580666 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.581048 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.581049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.581049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.581104 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.582426 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.582427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.582427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.582535 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.582692 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.582753 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.582857 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:15:38.583283 21412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:38.583601 21412 log.cc:824] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:38.584746 21412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: No bootstrap required, opened a new log
I20260809 16:15:38.584868 17282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:15:38.585978 21119 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "515360f971ad408c93e8cb0f4efd3faf"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59168
I20260809 16:15:38.586864 21412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
I20260809 16:15:38.587028 21412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350b00408fe9450c8a6679a0df360aba, State: Initialized, Role: FOLLOWER
I20260809 16:15:38.587116 21412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.190@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:15:38.587414 21421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.587484 21412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:38.587532 21421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:15:38.589488 21440 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:15:38.589535 21440 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:38.590552 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.582925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42848)
0809 16:15:38.582978 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.582990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.583027 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.584088 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.584089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.584247 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.584288 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.584289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.584906 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.584907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.585423 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.585425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.585812 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.585813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.585880 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.586281 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.586286 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.587736 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.589432 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.589432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.589433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.589643 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.589845 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.589845 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.589846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.589912 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.590194 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.590195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.590256 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.590361 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.590457 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:15:38.590559 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.583929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.16.224.188:38965)
0809 16:15:38.583969 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.583970 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.583978 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:15:38.583996 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.583997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.584005 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.584080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.584358 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.584813 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.585501 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.585734 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.585799 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.587817 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.587818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.587819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.587826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.589349 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.589711 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.589712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.589713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.589768 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.589981 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.589982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.589982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.590129 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.590195 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.590402 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.590481 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:38.589670 21304 master_runner.cc:186] Error getting master registration for 127.16.224.190:39995: OK, instance_id { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" instance_seqno: 1786292138241687 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:15:38.591867 21304 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:15:38.592087 21166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf: Bootstrap starting.
I20260809 16:15:38.592832 21166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:38.593128 21166 log.cc:824] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:38.593969 21166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf: No bootstrap required, opened a new log
I20260809 16:15:38.595383 21166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
I20260809 16:15:38.595527 21166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515360f971ad408c93e8cb0f4efd3faf, State: Initialized, Role: FOLLOWER
I20260809 16:15:38.595613 21166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
I20260809 16:15:38.595887 21443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.595937 21166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:38.595974 21119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:15:38.595988 21443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:38.596041 21119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:38.596649 21119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
I20260809 16:15:38.596853 21119 leader_election.cc:288] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [CANDIDATE]: Term 1 election: Requested vote from peers d00958ee0db642ffa117710072d62254 (127.16.224.189:39559), 350b00408fe9450c8a6679a0df360aba (127.16.224.188:38965)
W20260809 16:15:38.598055 21464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:15:38.598107 21464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:38.598039 21377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "515360f971ad408c93e8cb0f4efd3faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "350b00408fe9450c8a6679a0df360aba"
I20260809 16:15:38.598152 21377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:38.599370 21377 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 515360f971ad408c93e8cb0f4efd3faf in term 1.
I20260809 16:15:38.599860 21079 leader_election.cc:302] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350b00408fe9450c8a6679a0df360aba, 515360f971ad408c93e8cb0f4efd3faf; no voters:
I20260809 16:15:38.599975 21443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:15:38.600121 21443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [term 1 LEADER]: Becoming Leader. State: Replica: 515360f971ad408c93e8cb0f4efd3faf, State: Running, Role: LEADER
I20260809 16:15:38.600215 21443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
I20260809 16:15:38.600623 21443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 515360f971ad408c93e8cb0f4efd3faf. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.600698 21443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:38.600481 21047 master_runner.cc:186] Error getting master registration for 127.16.224.189:39559: OK, instance_id { permanent_uuid: "d00958ee0db642ffa117710072d62254" instance_seqno: 1786292138404003 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:15:38.602713 21047 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:15:38.606426 21482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:15:38.606652 21482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:15:38.609122 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.598498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.16.224.188:38965)
0809 16:15:38.598533 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.598535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.599081 (+ 546us) server_negotiation.cc:407] Connection header received
0809 16:15:38.599214 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.599215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.599226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.599253 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.600948 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.601642 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.603337 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.603629 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.603670 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.605829 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.605831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.605831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.605838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.607411 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.608008 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.608010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.608010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.608083 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.608402 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.608403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.608403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.608528 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.608599 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.608935 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.609032 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:15:38.610421 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.599509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.16.224.190:39995)
0809 16:15:38.599641 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.599643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.599650 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:38.599667 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.599668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.599677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.599732 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.603414 (+ 3682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.604422 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.605388 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.605679 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.605714 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.607507 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.607508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.607509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.607515 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.609018 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.609398 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.609399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.609400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.609453 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.609631 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.609631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.609632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.609745 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.609839 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.610220 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.610293 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 16:15:38.616344 21377 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [term 1 FOLLOWER]: Refusing update from remote peer 515360f971ad408c93e8cb0f4efd3faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:38.616695 21443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:38.617655 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.598736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.16.224.189:39559)
0809 16:15:38.598780 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.598781 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.609820 (+ 11039us) server_negotiation.cc:407] Connection header received
0809 16:15:38.609848 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.609849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.609860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.609903 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.611194 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.611965 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.612924 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.613245 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.613287 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.614700 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.614702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.614702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.614710 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.616252 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.616727 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.616728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.616729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.616787 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.617088 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.617089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.617089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.617221 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.617300 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.617478 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.617560 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:15:38.618158 21421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 515360f971ad408c93e8cb0f4efd3faf. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.618251 21421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:38.623986 21443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.624094 21443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:38.624104 21421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.624204 21421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:38.624414 21483 mvcc.cc:205] Tried to move back new op lower bound from 7316652599766487040 to 7316652599707172864. Current Snapshot: MvccSnapshot[applied={T|T < 7316652599766487040}]
I20260809 16:15:38.626369 21484 mvcc.cc:205] Tried to move back new op lower bound from 7316652599766487040 to 7316652599707172864. Current Snapshot: MvccSnapshot[applied={T|T < 7316652599766487040}]
I20260809 16:15:38.631237 21482 catalog_manager.cc:1347] Generated new cluster ID: dc42b600fde64a419d36ee5bbd52455b
I20260809 16:15:38.631290 21482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:15:38.636370 21284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } attempt: 3
Aug 09 16:15:38 dist-test-slave-fgb3 krb5kdc[21027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292138, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.224.189@KRBTEST.COM for kudu/127.16.224.188@KRBTEST.COM
I20260809 16:15:38.645525 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.636655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.224.188:38965 (local address 127.0.0.1:42866)
0809 16:15:38.636742 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 16:15:38.636754 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:15:38.636799 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:15:38.636997 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:15:38.636998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:15:38.637188 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:15:38.637234 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.637236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.638216 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.638217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.639063 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:15:38.639067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.639597 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.639598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:15:38.639656 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.640119 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:15:38.640124 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:15:38.641700 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:15:38.644336 (+ 2636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.644337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.644338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.644565 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.644809 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:15:38.644810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:15:38.644810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:15:38.644881 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:15:38.645208 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:15:38.645209 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:15:38.645270 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 16:15:38.645330 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:15:38.645426 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 16:15:38.646036 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:38.636697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.16.224.188:38965)
0809 16:15:38.636865 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.636867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.636873 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:15:38.636892 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.636893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.636902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.636944 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.637311 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.638100 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.639155 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:38.639460 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:38.639518 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:38.641799 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:15:38.641801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:15:38.641802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:15:38.641809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:15:38.643362 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.644657 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.644659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.644659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.644732 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:15:38.644966 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:15:38.644967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:15:38.644968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:15:38.645108 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:15:38.645165 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:38.645859 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 16:15:38.645937 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260809 16:15:38.648010 21284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: Bootstrap starting.
I20260809 16:15:38.648766 21284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: Neither blocks nor log segments found. Creating new log.
I20260809 16:15:38.649088 21284 log.cc:824] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: Log is configured to *not* fsync() on all Append() calls
I20260809 16:15:38.650238 21284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: No bootstrap required, opened a new log
I20260809 16:15:38.652325 21284 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
I20260809 16:15:38.652470 21284 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00958ee0db642ffa117710072d62254, State: Initialized, Role: FOLLOWER
I20260809 16:15:38.652550 21284 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } }
I20260809 16:15:38.652880 21284 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:15:38.654173 21238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "515360f971ad408c93e8cb0f4efd3faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d00958ee0db642ffa117710072d62254"
I20260809 16:15:38.654280 21238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:15:38.655133 21238 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 515360f971ad408c93e8cb0f4efd3faf in term 1.
I20260809 16:15:38.655341 21236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [term 1 FOLLOWER]: Refusing update from remote peer 515360f971ad408c93e8cb0f4efd3faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:15:38.656183 21488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:15:38.659715 21482 catalog_manager.cc:1370] Generated new certificate authority record
W20260809 16:15:38.660208 21517 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 16:15:38.660425 21482 catalog_manager.cc:1504] Loading token signing keys...
W20260809 16:15:38.660624 21517 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:15:38.660544 21495 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.660778 21495 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:38.660861 21495 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 515360f971ad408c93e8cb0f4efd3faf. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.660964 21495 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:38.661034 21495 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.661120 21495 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:15:38.664259 21480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "515360f971ad408c93e8cb0f4efd3faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "515360f971ad408c93e8cb0f4efd3faf" member_type: VOTER last_known_addr { host: "127.16.224.190" port: 39995 } } peers { permanent_uuid: "d00958ee0db642ffa117710072d62254" member_type: VOTER last_known_addr { host: "127.16.224.189" port: 39559 } } peers { permanent_uuid: "350b00408fe9450c8a6679a0df360aba" member_type: VOTER last_known_addr { host: "127.16.224.188" port: 38965 } } }
I20260809 16:15:38.664394 21480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf [sys.catalog]: This master's current role is: LEADER
I20260809 16:15:38.666656 21482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 515360f971ad408c93e8cb0f4efd3faf: Generated new TSK 0
I20260809 16:15:38.666826 21482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:15:38.667300 21520 mvcc.cc:205] Tried to move back new op lower bound from 7316652599766487040 to 7316652599707172864. Current Snapshot: MvccSnapshot[applied={T|T < 7316652599766487040}]
W20260809 16:15:38.699107 21416 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.697274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.16.224.188:38965)
0809 16:15:38.697336 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.697338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.698414 (+ 1076us) server_negotiation.cc:407] Connection header received
0809 16:15:38.698523 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.698525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.698536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.698565 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.699037 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42876: BlockingRecv error: recv got EOF from 127.0.0.1:42876 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:15:38.699899 21173 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.697704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.16.224.190:39995)
0809 16:15:38.697745 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.697747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.699371 (+ 1624us) server_negotiation.cc:407] Connection header received
0809 16:15:38.699470 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.699471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.699482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.699511 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.699834 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59190: BlockingRecv error: recv got EOF from 127.0.0.1:59190 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:15:38.700644 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.698037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.16.224.189:39559)
0809 16:15:38.698078 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.698080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.700130 (+ 2050us) server_negotiation.cc:407] Connection header received
0809 16:15:38.700222 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.700223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.700234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.700261 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.700580 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49120: BlockingRecv error: recv got EOF from 127.0.0.1:49120 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:15:38.702970 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.702177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.16.224.189:39559)
0809 16:15:38.702211 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.702213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.702456 (+ 243us) server_negotiation.cc:407] Connection header received
0809 16:15:38.702563 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.702564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.702582 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.702611 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.702924 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49130: BlockingRecv error: recv got EOF from 127.0.0.1:49130 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:15:38.705925 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.705166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.16.224.189:39559)
0809 16:15:38.705197 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.705199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.705448 (+ 249us) server_negotiation.cc:407] Connection header received
0809 16:15:38.705540 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.705541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.705551 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.705579 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.705882 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49144: BlockingRecv error: recv got EOF from 127.0.0.1:49144 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:15:38.709921 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.709067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.16.224.189:39559)
0809 16:15:38.709103 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.709105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.709378 (+ 273us) server_negotiation.cc:407] Connection header received
0809 16:15:38.709472 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.709473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.709485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.709511 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.709874 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49148: BlockingRecv error: recv got EOF from 127.0.0.1:49148 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:15:38.715868 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.715080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.16.224.189:39559)
0809 16:15:38.715117 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.715119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.715359 (+ 240us) server_negotiation.cc:407] Connection header received
0809 16:15:38.715449 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.715450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.715461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.715489 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.715817 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49152: BlockingRecv error: recv got EOF from 127.0.0.1:49152 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:15:38.725900 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.725033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.16.224.189:39559)
0809 16:15:38.725072 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.725074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.725341 (+ 267us) server_negotiation.cc:407] Connection header received
0809 16:15:38.725440 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.725441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.725453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.725480 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.725848 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49166: BlockingRecv error: recv got EOF from 127.0.0.1:49166 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:15:38.744185 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.743240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.16.224.189:39559)
0809 16:15:38.743284 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.743286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.743585 (+ 299us) server_negotiation.cc:407] Connection header received
0809 16:15:38.743689 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.743691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.743703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.743731 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.744122 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49168: BlockingRecv error: recv got EOF from 127.0.0.1:49168 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:15:38.778095 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.777398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.16.224.189:39559)
0809 16:15:38.777444 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.777446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.777545 (+ 99us) server_negotiation.cc:407] Connection header received
0809 16:15:38.777614 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.777615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.777627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.777679 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.778031 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49178: BlockingRecv error: recv got EOF from 127.0.0.1:49178 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:15:38.843890 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.843159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.16.224.189:39559)
0809 16:15:38.843234 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.843236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.843309 (+ 73us) server_negotiation.cc:407] Connection header received
0809 16:15:38.843381 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.843382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.843394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.843440 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.843825 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49188: BlockingRecv error: recv got EOF from 127.0.0.1:49188 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 16:15:38.973749 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:38.973136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.16.224.189:39559)
0809 16:15:38.973220 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:38.973222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:38.973235 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:15:38.973299 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:38.973300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:38.973312 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:38.973352 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:38.973689 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49198: BlockingRecv error: recv got EOF from 127.0.0.1:49198 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 16:15:39.231608 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:15:39.231010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.16.224.189:39559)
0809 16:15:39.231072 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:39.231074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:39.231085 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:15:39.231145 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:39.231146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:39.231158 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:39.231207 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:15:39.231546 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49210: BlockingRecv error: recv got EOF from 127.0.0.1:49210 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:15:39.589989 21440 catalog_manager.cc:1259] Loaded cluster ID: dc42b600fde64a419d36ee5bbd52455b
I20260809 16:15:39.590067 21440 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: loading cluster ID for follower catalog manager: success
I20260809 16:15:39.590871 21440 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: acquiring CA information for follower catalog manager: success
I20260809 16:15:39.591148 21440 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 350b00408fe9450c8a6679a0df360aba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:15:39.661084 21517 catalog_manager.cc:1259] Loaded cluster ID: dc42b600fde64a419d36ee5bbd52455b
I20260809 16:15:39.661156 21517 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: loading cluster ID for follower catalog manager: success
I20260809 16:15:39.661933 21517 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: acquiring CA information for follower catalog manager: success
I20260809 16:15:39.662204 21517 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d00958ee0db642ffa117710072d62254: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:15:39.747272 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:39.744876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.16.224.189:39559)
0809 16:15:39.744987 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:39.744989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:39.745025 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:15:39.745102 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:39.745103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:39.745160 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:39.745232 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:39.745382 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.745875 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.746661 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.746843 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.746886 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:39.747005 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:39.747118 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:39.747173 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:39.747188 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 16:15:39.747206 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":56,"threads_started":1}
I20260809 16:15:39.751216 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:39.748909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.16.224.188:38965)
0809 16:15:39.749040 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:39.749043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:39.749050 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:15:39.749085 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:39.749085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:39.749120 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:39.749186 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:39.749286 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.749757 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.750520 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.750715 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.750768 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:39.750910 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:39.751009 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:39.751075 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:39.751111 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:15:39.751132 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260809 16:15:39.755399 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:39.752782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.16.224.188:38965)
0809 16:15:39.752819 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:39.752821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:39.752881 (+ 60us) server_negotiation.cc:407] Connection header received
0809 16:15:39.752928 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:39.752929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:39.752939 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:39.752982 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:39.753125 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.753746 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.754692 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.754978 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.755045 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:39.755192 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:39.755223 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:39.755265 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:39.755312 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 16:15:39.755331 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:15:39.755842 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:39.753064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.16.224.189:39559)
0809 16:15:39.753105 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:39.753107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:39.753220 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:15:39.753295 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:39.753296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:39.753307 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:39.753353 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:39.753449 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.754178 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.755049 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.755334 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.755391 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:39.755575 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:39.755609 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:39.755634 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:39.755747 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 16:15:39.755775 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:15:39.756004 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:15:39.752732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.16.224.190:39995)
0809 16:15:39.752840 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 16:15:39.752842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:15:39.753040 (+ 198us) server_negotiation.cc:407] Connection header received
0809 16:15:39.753093 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:15:39.753094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:15:39.753146 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:15:39.753218 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:15:39.753334 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.754126 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.755022 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:15:39.755335 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:15:39.755399 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:15:39.755608 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:15:39.755712 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:15:39.755762 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:15:39.755900 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 16:15:39.755928 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":52,"threads_started":1}
I20260809 16:15:39.757334 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 21047
I20260809 16:15:39.774302 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 21179
I20260809 16:15:39.787933 17282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLoTDwK/build/release/bin/kudu with pid 21304
2026-08-09T16:15:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2917 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5868 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271288 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST